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FOREWORD 


The  objectives  of  this  evaluation  were  to  (1)  provide  or 
develop  user-level  methods  for  human  factors  evaluation  from 
source  selection  through  fielding  of  the  Non-Line-Of-Sight  (NLOS) 
Fiber-Optic  Guided  Missile  (FOG-M)  system;  (2)  develop  effective 
strategies  for  integrating  Manpower  and  Personnel  Integration 
(MANPRINT)  information  and  requirements  into  the  NLOS  FOG-M 
system  procurement  process;  and  (3)  demonstrate  MANPRINT  poten¬ 
tial  for  improvements  to  force  readiness,  system  operational 
availability,  fire  unit  probability  of  kill,  weapon  system  de¬ 
sign,  and  the  optimization  of  manpower,  personnel,  and  training 
resources. 

The  research  was  performed  by  the  Crew  Weapons  Performance 
Team  of  the  Fort  Bliss  Field  Unit  of  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI) .  This 
research  directly  supported  the  ARI  research  task,  “Soldier- 
System  Effectiveness  in  Air  Defense."  The  efforts  reported  here 
were  undertaken  at  the  invitation  of  the  U.S.  Army  Air  Defense 
Artillery  School  (USAADASCH) ,  the  NLOS  Training  and  Doctrine 
Command  (TRADOC)  system  manager,  and  the  Operational  Test  and 
Evaluation  Command  (OPTEC) . 

The  concerns  and  data  identified  during  the  course  of  this 
project  were  integrated  into  the  NLOS  research  and  development 
program  through  active  participation  with  the  NLOS  program  man¬ 
ager's  office,  the  NLOS  TRADOC  system  manager,  the  system  devel¬ 
oper,  the  System  Safety  and  Health  Hazards  working  groups,  and 
the  USAADASCH  Directorate  of  Training  Development. 


EDGAR  M.  JOHNSON 
Director 
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MANPRINT  SUPPORT  OF  THE  NON-LINE-OF-SIGHT  FIBER-OPTIC  GUIDED 
MISSILE  SYSTEM 

EXECUTIVE  SUMMARY _ _ _ 


Requirement: 

This  research  is  part  of  an  effort  to  develop  and  apply 
concepts,  methods,  and  tools  for  the  evaluation  of  MANPRINT  con¬ 
cerns  to  support  the  research  and  development  program  of  the  Non- 
Line-Of-Sight  (NLOS)  Fiber-Optic  Guided  Missile  (FOG-M)  system. 
The  three  broad  objectives  of  this  effort  were  to  (1)  provide  or 
develop  user-level  methods  for  human  factors  evaluation  from 
source  selection  through  fielding,  (2)  develop  effective  strate¬ 
gies  for  integrating  MANPRINT  information  and  requirements  into 
the  weapon  procurement  process,  and  (3)  demonstrate  the  MANPRINT 
potential  for  improvements  to  force  readiness,  system  operational 
availability,  fire  unit  probability  of  kill,  weapon  system  de¬ 
sign,  and  the  optimization  of  manpower,  personnel,  and  training 
resources . 


Procedure : 

MANPRINT  issue  evaluations  were  conducted  for  the  NLOS  pre- 
production  system  by  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI)  Fort  Bliss  Crew  Weapons  Per¬ 
formance  Team.  Within  the  NLOS  Force  Development  Test  and  Expe¬ 
rimentation  (FDTE)  evaluation,  the  Performance  Team  examined 
soldier  task  performance  and  surveyed  human  factors  issues  of 
system  design.  For  Extended  User  Employment  (EUE) ,  the  Perfor¬ 
mance  Team  carried  out  a  series  of  low-cost  evaluations  using  the 
pre-production  system  and  equipment  mock-ups.  For  the  NLOS  De¬ 
fense  Simulation  Network-Developmental  (SIMNET-D)  evaluation,  the 
Task  Force  raised  the  issue  of  NLOS  engagement  effectiveness 
given  target  cuing  time  delays  inherent  in  the  required  command 
and  control  system. 


Findings: 

Results  from  MANPRINT  evaluations  can  be  identified  that 
support  each  of  the  three  major  objectives.  The  NLOS  FDTE 
MANPRINT  effort  utilized  a  human  factors  evaluation  framework 
directly  linked  to  the  MIL  STD  1472  Human  Factors  Engineering 
(HFE)  guidelines  that  provided  a  systematic  and  comprehensive 
framework  for  evaluation  that  was  used  throughout  the  follow-on 
NLOS  evaluations.  The  issue  of  operator  workload  was  evaluated 
during  field  testing  using  the  NASA  Task  Load  Index  (TLX)  tool. 
EUE  was  conducted  as  a  series  of  separate  low-cost  follow-up 


vii 


evaluations  of  lessons  learned  from  NLOS  FDTE  and  of  proposed 
design  changes  examining  the  Gunner Console  design,  crew  emer¬ 
gency  egress,  the  embedded  trainer,  crew  drill  night  performance, 
and  the  vehicle  light  signature. 

The  EUE  MANPRINT  effort  demonstrated  the  ability  of  the 
MANPRINT  approach  to  influence  the  design  of  new  systems  as 
concerns  regarding  the  overcrowding  of  the  crew  compartment  and 
related  safety  issues  lead  to  the  relocation  of  some  equipment 
items  to  enhance  operator  safety.  The  MANPRINT  paper  and  pencil 
evaluation  of  command  and  control  delays  and  target  movement 
during  NLOS  engagements  demonstrated  the  ability  of  the  MANPRINT 
program  to  isolate  and  address  key  issues  of  system  operational 
effectiveness . 

The  MANPRINT  potential  contribution  to  improved  force  readi¬ 
ness  through  error  assessment  and  error  reduction  can  be  esti¬ 
mated  in  terms  of  dollars  through  some  very  simple  calculations. 
For  example,  from  the  NLOS  FDTE  evaluation,  a  6%  error  rate  for 
Launch  Control  tasks  was  identified,  while  program  plans  called 
for  the  acquisition  of  16,050  missiles  at  a  cost  of  roughly 
$50,000  each.  If  Launch  Control  errors  leading  to  mission  fail¬ 
ure  could  be  reduced  by  50%  through  training  or  redesign,  the 
waste  of  481  missiles  might  be  avoided,  with  a  cost  savings  of 
$24,075,000. 


Utilization  of  Findings: 

The  results  of  the  NLOS  FDTE  MANPRINT  evaluations  were 
incorporated  into  the  final  test  report  produced  by  the  TEXCOM 
Air  Defense  Artillery  Board,  Fort  Bliss.  The  results  of  the  EUE 
MANPRINT  evaluations  were  distributed  as  formal  memoranda  through 
the  NLOS  Training  and  Doctrine  Command  (TRADOC)  system  manager  to 
the  NLOS  program  manager,  the  NLOS  Human  Engineering  working 
group,  the  NLOS  System  Safety  working  group,  equipment  develop¬ 
ers,  and  other  agencies  involved  in  NLOS  development.  The  simple 
NLOS  target  movement  and  search  area  estimates  were  of  sufficient 
power  to  raise  the  target  search  issue  to  the  level  of  a  formal 
test  issue  for  the  NLOS  SIMNET-D  evaluation.  The  concern  that 
target  cuing  delays  inherent  in  the  required  target  cuing  command 
and  control  systems  could  greatly  reduce  NLOS  engagement  effec¬ 
tiveness  was  supported  by  the  SIMNET-D  test  data  for  target 
detection. 
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MANPRINT  SUPPORT  OF  THE  NON-LINE-OF-SIGHT 
FIBER-OPTIC  GUIDED  MISSILE  SYSTEM 


Introduction 


Background 

The  Non-Line-Of -Sight  (NLOS)  Fiber-Optic  Guided  Missile 
(FOG-M)  systea  was  developed  to  counter  the  increasing  threat 
posed  by  precision  guided  weapons  delivered  by  rotary  wing 
aircraft  masked  behind  terrain  features.  NLOS  FOG-M  uses  fiber¬ 
optic  cable  guided  missiles  to  engage  stationary  and  moving 
rotary  wing  and  ground  targets  masked  by  terrain  or  hidden  from 
direct  line-of-sight.  The  NLOS  system  requirements  are  contained 
in  the  Capstone  Required  Operational  Cap2d>ility  (ROC)  for  the 
Forward  Area  Air  Defense  System  (FAADS) ,  annex  I-NLOS  Component, 
29  October  1987.  Several  prototype  NLOS  systems  were  constructed 
to  explore  aspects  of  the  design  prior  to  soliciting  design 
proposals  from  industry.  Rotary  wing  and  ground  targets  were 
successfully  engaged  with  missiles  launched  from  the  prototype 
systems  in  live- fire  demonstrations,  and  the  prototypes 
participated  in  ntunerous  technical,  operational,  and  simulation 
system  tests  between  1988  and  1991. 

In  December  1988  the  Army  awarded  a  development  contract  for 
the  NLOS  system  following  a  technical  competition  in  which 
competitors  submitted  design  and  cost  proposals.  On  January  23, 
1991,  the  Army  terminated  the  initial  NLOS  development  effort  due 
to  projected  program  cost  growth  concerns  and  changing 
requirements  for  system  capabilities.  The  prograun  was  restarted 
in  March  of  1991  for  an  NLOS  that  could  serve  as  a  ground  and  air 
target  Combined  Arms  weapon  system  with  the  U.S.  Army  Infantry 
School  as  the  lead  proponent  for  the  research  and  development 
effort.  The  revised  schedule  (September  1992)  called  for  the 
first  Army  units  to  be  equipped  with  NLOS  in  1999.  In  a  related 
effort,  France  and  Germany  awarded  project  definition  contracts 
in  January  1992  for  a  fiber-optic  guided  missile  to  be  used 
against  high-value  ground  targets. 


System  Description 

The  NLOS  system  consists  of  a  launcher  with  six  missiles, 
gunner's  station,  and  land  navigation  system  mounted  on  the  high 
mobility  multipurpose  wheeled  vehicle  (HMMHV)  (see  Figure  1) . 

NLOS  is  operated  by  a  two-man  crew  from  the  16P  Military 
Occupational  Speciality  (MOS)  currently  assigned  to  the  Chapparal 
air  defense  system.  One  crewmember  acts  as  driver  and  performs 
radio  commiuiicatlons  tasks.  The  second  crewmember  is  the  gunner. 
The  NLOS  gunner  uses  the  gunner's  station  controls  and  displays 
located  in  the  HMMWV  crew  cab  to  enter  a  missile  flight  path  into 
the  system  computer  prior  to  missile  la\uich.  The  path 
automatically  guides  the  missile  to  a  designated  target  area. 
After  launch,  the  missile  seeker  provides  a  look-down  view  of  the 
battlefield  on  the  gunner's  console  display  via  the  fiber-optic 
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Figure  1 .  Non-Line-Of-Sight  (NLOS)  Fiber-Optic  Guided 
Missile  (FOG-M)  system. 
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data  link  while  simultaneously  transmitting  missile  guidance 
commands  from  the  gunner  station.  The  missile  seeker  is 
programmed  to  systematically  search  the  target  area  until  the 
gunner  selects  a  target  for  engagement.  The  gunner  has  the 
option  to  activate  an  automatic  tracker  or  manually  fly  the 
missile  into  the  target  using  a  joystick  for  guidance  control. 
The  NLOS  missile  is  approximately  six  feet  in  length  with  a 
weight  of  approximately  150  pounds.  A  drawing  of  the  FOG-M 
missile  appears  as  Figure  2. 


MANPRINT  Goals 

NLOS  Manpower  and  Personnel  Integration  (MANPRINT) 
evaluations  were  conducted  as  part  of  the  NLOS  system  acquisition 
effort.  MANPRINT  refers  to  the  comprehensive  management  and 
technical  effort  to  ensure  total  system  effectiveness  by  the 
early  and  continuous  integration  into  materiel  development  and 
acquisition  of  all  relevant  Information  concerning  the  following 
domains:  (1)  manpower,  (2)  personnel,  (3)  training,  (4)  human 
engineering,  (5)  system  safety,  and  (6)  health  hazards.  The 
MANPRNT  progreun  is  described  in  Army  Regulation  602-2  (Manpower 
and  Personnel  Integration  [MANPRINT]  in  the  Materiel  Acquisition 
Process) .  The  Army  Research  Institute  (ARI)  Fort  Bliss  Field 
Unit  assisted  in  the  implementation  of  the  MANPRINT  program  by 
developing  and  applying  concepts,  methods,  and  tools  to  analyze 
MANPRINT  issues.  Specifically,  the  ARI  Crew  Weapons  Performance 
Team  at  Fort  Bliss  focused  on  three  basic  aspects  of  MANPRINT 
research  and  development: 

(1)  Provide  and  implement  user-level  methods  for  human 
performance  and  hximan  factors  evaluation,  from  source  selection 
through  fielding, 

(2)  Develop  effective  strategies  for  integrating  MANPRINT 
Information  and  requirements  into  the  weapon  procurement  process, 

(3)  Demonstrate  MANPRINT  potential  for  improvements  to  force 
readiness,  system  operational  availability,  fire  unit  probability 
of  kill,  weapon  system  design,  and  manpower,  personnel,  and 
training. 

From  1989  through  1991  the  ARI  Fort  Bliss  Field  Unit 
supported  MANPRINT  evaluations  during  the  NLOS  Force  Development 
Test  and  Experimentation  (FDTE) ,  the  Extended  User  Employment 
(EUE)  program,  and  a  Defense  Simulation  Network-Developmental 
(SIMNET-D)  MANPRINT  effort.  Efforts  were  directed  at  developing 
concepts,  tools,  and  methods  for  "soldier  in-the-loop" 
evaluations.  Lessons  learned  from  these  evaluations  provided 
decision  makers  with  information  on  key  concerns  to  support  the 
source  selection  evaluation  process,  the  acquisition  program 
milestone  decisions,  and  the  NLOS  Critical  Design  Review  decision 
process.  MANPRINT  findings  were  conveyed  through  test  reports, 
briefings,  memoranda  and  working  groups. 
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Figure  2.  Fiber-Optic  Guided  Missile  (FOG-M)  configuration. 


The  NLOS  MANPRINT  support  effort  was  designed  to  meet  the 
information  needs  of  the  research  and  development  effort  while 
working  within  the  constraints  of  the  limited  resources 
available.  Whenever  possible^  the  NLOS  MANPRINT  evaluations  were 
of  short  duration  and  conducted  during  normal  duty  hours  using 
available  equipment  and  soldiers.  The  goal  of  designing  usable 
systems  is  not  unique  to  MANPRINT.  A  number  of  authors  have 
identified  methods  for  developing  usable  systems  that  share 
elements  of  the  MANPRINT  philosophy  and  can  suggest  useful 
approaches  for  MANPRINT  evaluations.  Key  issues  in  the 
literature  include  the  relative  contributions  of  quantitative 
versus  qualitative  data,  the  need  for  examining  user  behavior  in 
the  task  environment,  the  use  of  mock-ups  in  design  evaluations, 
the  role  of  theories  of  cognition  in  system  design,  and  the 
adequacy  of  small  sample  sizes  for  gathering  user  feedback.  The 
NLOS  system  presents  the  operator  with  a  sophisticated  human- 
computer  interface  (HCI) ;  it  was  reasonable  to  consider  design 
evaluation  guidelines  and  recommendations  from  the  field  of  HCI 
research  in  deciding  on  an  approach  to  NLOS  MANPRINT  evaluation 
support . 

The  NLOS  MANPRINT  effort  sought  to  support  the  information 
needs  for  an  evolving  system  design  with  evaluations  that 
frequently  yielded  qualitative  data.  Booth  and  Marshall  (1989) 
separate  design  evaluation  efforts  into  formative  and  summative 
evaluations,  where  formative  evaluation  is  aimed  at  helping  the 
designer  to  refine  and  form  the  design  and  primarily  uses 
qualitative  data.  Summative  evaluations  typically  involve 
quantitative  information  based  on  overall  scores  on  particular 
measures.  The  authors  contend  that  these  scores,  regardless  of 
how  they  are  derived,  are  unlikely  to  tell  designers  what  should 
be  done  to  improve  the  design.  Booth  and  Marshall  (1989)  argue 
that  designers  will  need  to  know  more  about  why  errors  or 
misunderstandings  occurred  rather  than  just  their  absolute 
numbers.  In  the  later  stages  of  design,  quantitative  data  plays 
a  greater  role  as  designers  assess  the  usefulness  of  particular 
changes  to  a  mature  system  design. 

A  primary  information  goal  for  the  NLOS  MANPRINT  support 
effort  was  to  understand  operator  tasks  as  they  exist  in  an 
operational  military  environment.  From  the  HCI  usability 
literature,  Whiteside,  Bennett,  and  Holtzblatt  (1988)  identify  the 
limitations  of  studying  benchmark  tasks  in  a  laboratory  setting 
and  argue  for  a  "Contextual  Research"  approach  to  usability 
engineering  that  includes  design  context  factors  such  as  the  work 
environment,  time  demands,  motivational  factors,  and  the  need  to 
focus  on  uncovering  the  user's  personal  experience  of  the  system 
interface.  For  NLOS  "Contextual  Research"  means  exploring  the 
system  during  the  day,  at  night,  and  with  soldiers  wearing 
chemical  and  protective  suits,  and  also  examining  how  the  soldier 
and  equipment  function  when  things  go  wrong.  The  MANPRINT 
evaluations  follow  the  Whiteside  et  al.  (1988)  recommendations  to 


have  soldiers  "think  aloud"  during  task  performance  and  relies  on 
videotape  recordings  as  a  low  cost  and  flexible  data  collection 
technique  to  understand  the  nature  of  the  users'  work  experience. 

The  NLOS  MANPRINT  evaluation  made  extensive  use  of  mock-ups 
of  proposed  equipment  designs  to  help  users  recognize  design 
issues  and  identify  improvements.  From  the  field  of  HCI 
usability  engineering,  Carroll  and  Rosson  (1985)  argue  that  the 
most  important  aspect  of  empirical  approaches  such  as  iterative 
trials  with  mock-ups  is  that  it  encourages  the  discovery  of 
design  solutions  that  might  have  been  missed  in  a  purely  analytic 
approach.  In  particular,  Landauer  (1991)  argues  that  theories  of 
cognition  will  always  be  too  weak  to  play  an  important  role  in 
areas  such  as  HCI  design.  They  will  not  be  able  to  identify 
system  design  characteristics  more  effectively  than  empirical 
methods  of  task  and  performance  analysis  and  simulation  coupled 
with  foirmative  design  evaluations.  This  is  in  line  with  the 
Whiteside  et  al.  (1988)  call  for  "Contextual  Research."  Landauer 
(1991)  believes  that  design  considerations  are  extremely 
sensitive  to  work  context  and  characteristics  of  the  users,  and 
that  task  performance  simply  involves  too  many  variables  and  is 
too  complex  to  lend  itself  to  calculations.  Landauer  (1991) 
argues  for  the  use  of  simulations  accomplished  by  simple 
calculations  of  task  data  as  a  good  source  of  information  needed 
for  design  work.  The  NLOS  MANPRINT  effort  used  this  approach  to 
estimate  the  time  needed  for  target  engagements,  and  to  compare 
this  to  the  resulting  opportunity  a  target  has  to  move. 

A  final  (question  to  consider  in  presenting  the  NLOS  MANPRINT 
support  approach  is  that  the  evaluations  were  typically  conducted 
with  only  four  to  six  trained  NLOS  crewmembers  performing  a 
limited  number  of  task  trials.  For  the  purposes  of  gathering 
soldier  feedback  during  the  development  of  the  NLOS  system 
design,  this  small  sample  of  trained  personnel  should  be 
adequate.  Exploring  the  question  of  how  many  subjects  are  enough 
for  usability  evaluations,  Virzi  (1992)  found  that  (1)  80%  of 
usability  problems  are  detected  with  four  or  five  subjects,  (2) 
additional  subjects  are  less  likely  to  reveal  new  information, 
and  (3)  the  most  severe  usability  problems  are  likely  to  be 
detected  with  the  first  few  subjects.  Likewise,  Landauer  (1991) 
argues  that  "tests  with  2  to  10  users  usually  reveal  most  glaring 
flaws,  and  sometimes  offer  strong  guidance  for  positive 
improvements . " 

In  summary,  the  NLOS  MANPRINT  support  approach  used  small 
groups  of  soldiers  and  equipment  mock-ups.  This  method  made  it 
possible  to  quickly  generate  soldier  feedback  on  system  design 
Issues  to  satisfy  the  formative  information  needs  of  the 
equipment  developers. 
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'  NANPRINT  In  Force  Development  Test  and  Experimentation 


FDTE  Overview 

The  NLOS  FDTE  was  conducted  18  April  through  22  May  1989  at 
the  North  Oscura  Range  area  of  White  Sands  Missile  Range,  New 
Mexico.  The  FDTE  employed  a  single  preproduction  prototype  NI/>S 
system  for  the  test.  The  primary  objectives  of  the  FDTE  were  to 
observe  operator  performance  in  a  tactical  environment,  refine 
NLOS  crew  drills,  provide  inputs  for  optimizing  system 
effectiveness,  address  concepts  pertaining  to  the  interface  vl*^ 
the  manual  command  and  control  center,  and  assist  in  establishing 
a  baseline  for  the  NLOS  production  systems.  Since  the  NLOS 
system  simulators/ trainers  were  not  avall2d)le  for  FDTE,  and  no 
live  missile  firings  were  scheduled,  none  of  the  engagement  or 
training  device  criteria  could  be  addressed  during  FDTE.  Two  key 
test  Issues  were  identified  for  NLOS  FDTE  which  provided  an 
opportunity  to  address  MANPRINT  concerns: 

Test  Issue  1:  Do  the  individual  and  collective  tasks,  crew 
drills,  and  TTPs  prepare  the  NLOS  squad  to  optimize  the  system's 
performance? 

Test  Issue  2:  Can  representative  soldiers  use  ecpiipment 
provided  at  the  squad  level  to  support  the  NLOS  air  defense 
missions? 


MANPRINT  Integration.  ARI's  contribution  to  FDTE  began  long 
before  the  start  of  testing  through  participation  in  the  Required 
Operational  Characteristics  (ROC)  document  development,  the 
MANPRINT  Joint  Working  Group,  NLOS  acquisition  program  working 
groups,  and  the  NLOS  Test  Integration  Working  Group.  Working 
within  the  established  system,  the  teeua  was  able  to  integrate 
NANPRINT  SMHP  issues  and  methods  into  the  NLOS  FDTE  test  plan. 
Team  members  served  as  the  Army's  Contracting  Officer's 
Representative  in  managing  the  efforts  of  the  civilian  contractor 
teauB  gathering  and  analyzing  FDTE  MANPRINT  data.  ARI  teeun 
members  identified  the  MANPRINT  test  issues,  data  requirements, 
and  data  collection  and  reduction  methodology  for  use  at  the 
White  Sands  Missile  Range  during  testing,  and  in  the  centralized 
Data  Analysis  Group  (DAG)  at  Fort  Bliss. 


MANPRINT  Concerns 

The  intent  of  Test  Issue  1  was  to  evaluate  NLOS  system 
performance  for  eleven  specific  crew  drills.  The  NANPRINT 
evaluation  focused  on  operator  crew  drill  task  performance  to 
include  a  time  and  error  performance  evaluation,  error  "cause" 
assessment,  and  an  examination  of  the  adequacy  of  the  proposed 
two-man  teeun. 
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The  intent  of  Test  Issue  2  was  to  isolate,  document,  and 
correct  any  personnel  or  equipment  deficiencies  that  affected 
mission  performance.  The  MANPRINT  evaluation  focused  on  the 
extent  to  which  the  NLOS  system  conformed  to  the  principles  of 
good  human  factors  design  for  military  systems.  The  evaluation 
included  the  identification  of  potential  system  safety  and  health 
hazards.  MANPRINT  calls  for  the  early  and  continuous  evaluation 
of  soldier-oriented  design  issues  throughout  the  acquisition  and 
development  of  new  systems.  The  MANPRINT  teeun  investigation  of 
system  design  features  and  their  impact  on  performance  was  driven 
by  the  desire  to  demonstrate  the  value  of  examining  design 
concerns  as  early  as  possible  in  the  acquisition  process,  where 
system  design  flexibility  is  greatest. 


Method 

Method  Overview.  MANPRINT  FDTE  data  collection  and 
evaluation  involved  integrating  information  from  a  number  of 
sources.  "MANPRINT  cameras"  captured  soldier  task  performance 
video  data,  while  field  data  collectors  recorded  significant 
events  for  each  crew  drill.  An  on-site  debriefing  of  the  NLOS 
crews  after  each  day's  missions  was  used  to  further  explore  and 
identify  Important  aspects  of  task  performance,  to  include  the 
causes  associated  with  any  performance  errors.  At  Ft.  Bliss,  a 
MANPRINT  teeun  exeuained  and  scored  crew  drill  task  performance 
based  on  multiple  sources  of  crew  compartment  video,  an  automated 
data  bus  instrumentation  system  that  captured  control  activation 
times,  and  field  debriefing  information,  to  support  the  FDTE 
"Test-Fix-Test"  crew  drill  development  process.  A  subjective 
workload  assessment  was  also  conducted  after  crews  performed  test 
trials.  At  the  conclusion  of  field  testing,  a  human  factors 
review  was  conducted  with  crewmembers  using  a  structured 
interview  guide. 

MANPRINT  camera  task  performance  video  data.  The  video 
Information  presented  on  the  NLOS  Gunner's  Console  was  linked 
directly  by  hardwire  to  a  recorder,  while  two  "over-the-shoulder" 
cameras  captured  the  operators'  interactions  with  controls  and 
displays  (both  audio  and  video) .  Video  monitors  located  nearby 
allowed  test  personnel  to  follow  soldier  performance  and  record 
necessary  data  without  intruding  on  the  conduct  of  the  crew 
drill. 

On-site  crew  drill  scoring.  Field  analysts  were  present 
during  the  course  of  each  crew  drill  to  assess  the  effectiveness 
of  a  drill  and  to  note  any  factors  which  might  have  influenced 
performance.  Performance  information  was  recorded  on  a  "Crew 
Drill  Sheet"  tailored  to  the  unique  tasks  of  each  crew  drill. 

On-site  crew  debrief.  Field  analysts  conducted  an  on-site 
debrief  of  crews  after  each  day's  field  trials,  reviewing 
selected  portions  of  the  day's  videotapes  to  explore  concerns 
with  crew  drill  performance  and  any  observed  task  performance 
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errors.  The  debrief  provided  a  great  source  of  information  used 
in  determining  the  causes  of  crew  drill  errors. 

DAG  error  cause  coding.  The  videotape,  automated 
instrumentation,  and  field  notes  for  each  crew  drill  were 
forwarded  to  a  central  DAG  for  evaluation,  review  by  the  "scoring 
group",  and  entry  as  validated  data  into  the  test  data  base. 

Where  task  perfonnance  errors  were  identified,  the  MANPRINT  DAG 
teeuB  reached  consensus  on  the  nature  of  an  error  and  assigned  a 
"MANPRINT  Error"  code  to  indicate  the  most  likely  MANPRINT  domain 
affected,  e.g..  System  Safety,  Human  Factors,  Manpower.  To 
systematically  describe  the  nature  of  the  problem,  one  of  the 
following  error  description  codes  was  assigned  to  each  error: 
"Procedure  Omitted",  "Procedure  Inaccurate",  "Wrong  Time"  (task 
performed  out  of  order) ,  "Wrong  Procedure" ,  or  "Extra  Procedure 
Performed" . 

Hvunan  factors  evaluation.  The  prototype  NIOS  system  was 
examined  to  assess  how  well  it  conformed  to  the  principles  of 
good  human  factors  design.  A  structured  crew  and  Subject  Matter 
Expert  (SME)  debriefing  was  used.  Initial  guidance  for  the 
evaluation  was  provided  by  the  Human  Engineering  Led^oratory  (HEL) 
Field  Office  at  Fort  Bliss.  Guidelines  for  the  evaluation  were 
obtained  from  two  documents:  TECOM  Hiunan  Factors  Engineering 
Guidelines  and  H\im2ui  Factors  Engineering  Data  Guide  for 
Evaluation  (HEDGE)  (Test  Operations  Procedure  1*2>610,  parts  1 
and  2,  1983)  and  TECOM  Soldier-Computer  Interface  Guidelines 
(Test  Operations  Procedure  1-1-059,  1985).  These  guidelines  were 
used  to  produce  a  twenty-page  structured  interview  docximent  for 
gathering  data  that  would  meet  the  information  demands  of  the 
Human  Engineering  Design  Criteria  for  Military  Systems,  Equipment 
and  Facilities,  MIL-STD-1472,  1989.  The  five  major  areas 
identified  for  the  NLOS  analysis  were:  operator  workspace, 
displays,  controls,  soldier-computer  interface,  and  the  missile 
reload  equipment.  Soldier  and  SME  responses  were  recorded  on  the 
Interview  guide  and  included  in  the  NLOS  FDTE  test  report  in 
full,  as  well  as  being  summarized  in  a  t£d}le. 

Workload  Assessment.  Crew  workload  was  a  key  concern  in  the 
NLOS  FDTE,  particularly  where  crew  drills  required  substantial 
physical  exertion  while  wearing  the  Mission  Oriented  Protective 
Posture  level  4  (MOPP  4)  chemical  protective  suits.  The  Task 
Load  Index  (TLX)  subjective  workload  rating  scale  (NASA-Ames 
Research  Center,  1986)  was  used  to  gather  soldier  ratings  of  the 
workload  involved  for  the  eleven  separate  drills  which  make  up 
the  total  NLOS  mission  (see  Figure  3) .  The  TLX  rating  scale  was 
administered  to  crewmembers  at  the  test  site  immediately 
following  a  mission  which  lasted  approximately  four  hours  and 
consisted  of  several  drills  (l.e..  System  Power  Up,  Emplacement, 
March  Order) .  The  ratings  data  were  not  subjected  to  a  paired- 
comparison  sozrt  procedure  because  the  work  of  Byers,  Bittner,  and 
Hill  (1989)  strongly  suggests  that  this  procedure  is  unnecessary. 
Instead,  the  average  of  the  crewmembers  raw  ratings  on  each  of 
the  six  TLX  factors  was  used. 
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1  •  DAY  MOPP  0 

2  -  DAY  MOPP  4 

3  •  NIGHT  MOPP  0 

4  >  NIGHT  MOPP  4 


TLX  Rating  Scales 


Task  or  Mission  Segment: _ 

Please  rate  the  task  or  mission  segment  by  putting  mark  on  each  of  the  six 
scales  at  the  point  which  matches  your  experience. 

or::  hhiiiih  ih  1 1  i 


Very  Low 

(HOW  MENTALLY  DEMANDING  WAS  THE  TASK?) 

Physical 

Demand  _LJ _ L_L.J _ L 

Very  Low 

(HOW  PHYSIC  ALLY  DEMANDING  WAS  THE  TASK?) 

Temporal 

Demand  |  |  |  |  I  I  I  I  I  I  I 

Very  Low 

(HOW  HURRIED  OR  RUSHED  WAS  THE  PACE  OF  THE  TASK?) 

Performance  I  i  1  I  I  I  I 


Very  High 


Very  High 


Very  High 


Perfect 


Failure 


(HOW  SUCCCESSFUL  WERE  YOU  IN  ACCOMPLISHING  WHAT  YOU  WERE  ASKED  TO  DO?) 

I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I  I 

Very  Low  Very  High 

(HOW  HARD  DID  YOU  HAVE  TO  WORK  TO  ACCOMPLISH  YOUR  LEVEL  OF  PERFORMANCE?) 

Frustration  I  ■  I  •  I  ■  I  .  I  .  I  .  I  .  I  .  I  .  I  .  I 


Very  Low 


(HOW  INSECURE,  DISCOURAGED,  IRRITATED.  AND  ANNOYED  WERE  YOU?) 


Very  High 


Figure  3.  Task  Load  Index  (TLX)  rating  scales. 
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Test  Issue  1.  Issue  1  addressed  crew  drill  task  performance 
assessment:  performance  evaluation,  error  assessment,  and  the 
adequacy  of  the  two-man  squad.  NLOS  system  performance  was 
evaluated  for  eleven  specific  crew  drills.  Results  of  the 
MANPRINT  evaluations  were  incorporated  into  the  NLOS  FDTE  Test 
Report  (Matthews,  Keaton,  Jolly,  Lewis,  Sanders,  and  Griffith, 
1989)  . 

The  MANPRINT  crew  drill  error  analysis  is  summarized  in 
Table  1,  with  a  total  of  478  errors  being  identified  overall. 
Multiplying  the  number  of  steps  in  a  drill  by  the  number  of  times 
the  drill  was  performed  yields  the  total  opportunities  for  error 
in  that  drill.  As  example.  System  Power  Up  has  30  steps  and  was 
performed  37  times  for  a  total  of  1110  opportunities  for  error. 
Performance  of  the  eleven  crew  drills  yielded  a  total  of  5,009 
drill  steps  in  which  an  error  could  occur.  The  total  of  478 
errors  represents  an  overall  error  rate  of  2d>out  9  percent.  The 
proposed  two-man  teeua  was  thus  capable  of  accomplishing 
approximately  91  percent  of  the  crew  drill  tasks  to  the 
prescribed  standards.  Examining  the  errors  which  occured, 
approximately  40  percent  were  attributed  to  procedures  being 
omitted,  while  procedures  performed  inaccurately  or  at  the  wrong 
time  accounted  for  22  and  27  percent  of  all  errors,  respectively. 
Where  performance  errors  occured  they  were  also  classified  in 
terms  of  which  of  the  six  MANPRINT  domains  (e.g.,  system  safety, 
health  hazards)  might  be  the  most  likely  cause. 

There  were  610  performance  errors.  Analysis  revealed  133 
errors  to  be  associated  with  the  conduct  of  the  task  and  477 
errors  with  MANPRINT  domains.  Ninety-seven  percent  (465)  of 
these  latter  errors  were  attributed  to  the  training  domain.  The 
near  total  attribution  of  errors  to  training  comes  in  part  from 
the  "Test-Flx-Test"  approach  to  the  FDTE  evaluation,  where  tasks 
and  task  sequencing  were  subject  to  frequent  changes  during  the 
course  of  testing  which  limited  the  the  opportunities  for 
retraining  crews.  A  problem  with  this  error  cause  coding 
approach  is  that  it  cannot  be  determined  whether  training  does  in 
fact  underlie  the  observed  errors.  There  is  no  way  of  knowing  if 
the  errors  might  persist  after  tasks  and  training  have  been 
steJsalized.  If  errors  did  persist,  this  would  suggest  that  some 
other  aspect  of  the  system  is  the  source  of  performance  errors. 

Test  Issue  2.  Issue  2  exeu&ined  NLOS  system  conformity  to 
the  principles  of  good  human  factors  design  for  military  systems, 
particularly  with  respect  to  the  issue  of  man-machine  interface 
under  the  conditions  imposed  in  an  operational  environment.  This 
was  accomplished  through  a  hands-on  review  of  the  system  and 
structured  crew  and  SME  debriefs.  The  five  major  areas 
Identified  for  analysis  were:  operator  workspace,  displays, 
controls,  soldier-computer  interface,  and  the  missile  reload 
equipment.  An  overview  of  the  Human  Factors  MANPRINT  domain 
concerns  Identified  is  presented  in  Table  2.  One  advantage  of 
esteUslishing  a  MIL-STD-1472-based  evaluation  framework  was  that 
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Crew  Drill  Error  Analysis 
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Table  2 

Human  Factors  Design  Review 


Hiunan  Factors  Element 

Crew  and  SME  Interviews 

1 .  Operator  Workspace 

o  Workspace  Configuration 

o  Seating 

o  Crew  Communications 

o  Work  Environment 

o  Driver  space  adequate;  gunner 
space  cramped;  needs  rifle  and 
tool  storage,  writing  surfaces 

o  No  gunner's  seat  adjustment 

o  Limitations  when  off-vehicle,  in 
M0PP4 

o  Crew  cab  heat,  toxic  fumes 
when  under  camouflage,  needs 
crew  cab  and  reload  illumination 

2 .  Controls 

o  Dimensions 

o  Resistance 

o  Location 

o  Good  control  grip  speed  and 
accuracy 

o  Good  Programmable  Display 

Pushbutton  (PDP)  tactile  response 
o  Good  view  and  reach  of 
pushbuttons  and  switches 

3 .  Displays 

o  Size,  Viewing  Distance, 
and  Angle 

o  Document  Holders 

o  Display  Format  and 

Content 

o  Size  adequate,  screen  glare  a 
problem,  forward  lean  observed 
o  Target  data  display  (used 
windows,  map,  paper) 
o  Call  for  fire  message  vs.  target 
data  display 

4 .  Soldier-Computer  Interface 

o  Data  Entry  Procedures 

o  Interactive  Control 

o  Feedback 

o  Error  Management/ Data 
Protection 

o  Entries  validated,  indicates 
acceptance ,  delays 
o  Good  menus,  data  entries 
echoed  on  screen 
o  Good  prompts,  error  messages, 
default  values 

o  Adequate  error  correction  in 
menus,  error  messages 
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Table  2 

Human  Factors  Design  Review  (cont.) 


Human  Factors  Element 

Crew  and  SME  Interviews 

5.  Missile  Reload  Equipment 

o  Reach,  Lift,  Push,  Pull 
Tasks 

o  Step  Points  and  Handholds 

o  Fasteners  and  Connectors 

o  Operating  Environment 

o  No  lifting,  moving  problems  cited, 
some  reach  limitations 
o  Improve  step  and  handholds, 
canister  handholds 
o  Canister  hoist  lug,  rail  and  pin 
tolerances,  location  of  5331 
connector,  six-pack  hoist  lugs 
o  Night  reload  needs  illumination 

this  perspective  results  in  identifying  vhat  is  good  edsout  the 
system  design  as  well  as  any  potential  deficiencies,  so  that  the 
Human  Factors  review  goes  beyond  a  simple  list  of  complaints. 
Giving  the  design  a  full  review  to  identify  both  strengths  and 
weaknesses  makes  this  evaluation  more  acceptedsle  to  the  equipment 
developers  as  a  starting  point  for  the  discussion  of  concerns. 

Many  of  the  Human  Factors  MANPRINT  concerns  identified  early 
in  FDTE  would  remain  Issues  throughout  the  NLOS  development 
progr2uii.  Lack  of  gunner  work  space  was  identified  as  a  concern 
and  set  the  stage  for  later  issues  when  equipment  additions  and  a 
Gunner's  Console  redesign  were  proposed.  Crew  cab  heat  remained 
a  design  issue  of  major  proportion  throughout  development.  On¬ 
site  MANPRINT  data  collectors  noted  that  the  NLOS  crews  would  use 
scraps  of  paper,  and  even  the  vehicle  windows,  as  a  surface  on 
which  to  record  critical  target  data  messages,  suggesting  yet 
another  design  inadequacy.  This  observation  was  recorded  as  a 
"Data  Display”  concern.  The  review  of  the  missile  reload 
equipment  led  to  the  identification  of  simple  or  common  sense 
design  Issues,  such  as  the  need  for  steps  and  handholds  for  on- 
vehlcle  tasks.  The  MANPRINT  team  pointed  out  that  the  conduct  of 
FDTE  night  operations  was  flawed  by  providing  crews  with 
supplementary  lighting  sources  to  meet  task  illvimination 
requirements.  System  Safety  and  Health  Hazard  MANPRINT  domain 
concerns  identified  during  the  evaluation  Include  excessive  heat 
and  vehicle  exhaust  fumes  in  the  crew  compartment,  and  concerns 
about  equipment  design  and  procedures  used  in  single  missile 
reload. 

Results  of  the  TLX  workload  evaluation  were  consistent  with 
the  crew  debrief  results.  From  the  crew  debrief,  crewmembers 
stated  that  the  workload  required  during  single  missile  reloads 
and  particularly  the  unloading  of  full  weight  missile  canisters 
using  the  prototype  missile  reload  equipment  was  excessive.  The 
average  of  the  six  TLX  subjective  workload  ratings  for  the  single 
missile  reload  battle  drill  (32.47)  was  significantly  higher  than 
the  average  of  the  remaining  nine  non-reload  drills  (11.74) 
[£(1,30)  «  45.8,  p  <.0001].  Likewise  the  average  rating  for 
multiple  missile  reload  (23.38)  was  significantly  greater  than 
the  average  for  the  remaining  nine  non-reload  drills  (11.74) 
[£(1,30)  «  8.0,  B<.001].  While  the  relative  level  of  workload 
for  the  two  reload  drills  was  greater  than  that  for  the  other 
drills,  the  means  for  all  the  drills  were  well  below  the  scale 
value  of  50  which  marks  the  center  point  on  the  rating  scale  (see 
Table  3). 

gonglMSiong; _ FDTE  Analysis 

The  FDTE  analysis  was  a  valuable  means  of  capturing  soldier- 
oriented  information  early  on  in  system  development.  Performance 
scoring  attempted  to  go  beyond  the  tallying  of  errors  to  include 
the  identification  of  the  nature  and  source  of  errors  encountered 
through  the  use  of  error  "cause  codes”  on  scoring  sheets,  but 
this  approach  met  with  only  limited  success.  The 
score  sheets  were  a  good  means  of  recording  the  behaviors 
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Table  3 

Task  Load  Index  (TLX)  Subjective  Workload  Estimates 
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1/  How  demanding  was  the  task?  0  =  Very  Low  Demands,  1 00  =  Very  High 

2!  How  successful  were  you  in  accomplishing  what  you  were  asked  to  do?  0  =  Perfect  Performance,  100  =  Failure 


associated  with  errors,  but  the  cause  of  the  error  was  almost 
always  Identified  as  training,  which  could  not  be  confirmed. 
Scoring  of  performance  errors  attributed  76  percent  of  all  errors 
to  training  causes,  and  22  percent  to  test  conduct  related 
problems  not  associated  with  a  particular  MANPRINT  domain.  While 
the  errors  could  be  the  result  of  the  frequent  changes  introduced 
to  the  drills  and  crewmembers  simply  forgetting  what  the 
currently  "correct"  procedure  was  it  is  not  possible  to  irule  out 
other  potential  causes,  such  as  system  design  features  or 
manpower  resources.  Approximately  40  percent  of  all  task 
performance  errors  were  due  to  procedures  being  omitted.  In  most 
cases,  these  omissions  were  not  critical,  such  as  a  crewman's 
failure  to  make  a  redundant  vocal  announcement  to  indicate  that 
he  had  performed  a  specific  task. 

While  the  error  cause  coding  scheme  did  not  generate  useful 
information,  the  MANPRINT  paragraph  summarizing  performance  for 
each  crew  drill  was  extreemly  valuedsle  in  providing  the  reader 
with  a  plain  english  description  of  what  occured.  The 
quantitative  time  and  error  performance  data  presented  in  the 
FDTE  Test  Report  could  often  hide  very  simple  explanations  of 
what  crewmembers  actually  did  to  perform  their  tasks.  For 
example,  in  the  single  missile  reload  drill,  crewmembers 
consistently  committed  a  high  number  of  task  performance  errors 
according  to  the  test  criteria.  Upon  ex2uiiination  it  was  found 
that  crewmembers  were  simply  performing  their  tasks  in  a  more 
logical  and  efficient  order  than  that  prescribed  in  the  drills 
they  were  trained  on.  What  was  really  needed  was  a  change  in  the 
drill,  not  the  soldier  task  performance. 

The  human  factors  review  identified  a  number  of  concerns 
which  were  investigated  in  follow-on  NLOS  evaluations.  The  focus 
on  displays,  controls,  workspace,  soldier-computer  interface, 
missile  reload  equipment  concerns,  and  reliance  on  MIL-STD-1472 
human  factors  engineering  guidelines  remained  throughout  NLOS 
testing.  From  the  operator  workspace  review,  a  lack  of  workspace 
within  the  crew  cab  and  concerns  eibout  cab  heat  were  noted.  It 
was  noted  that  the  conduct  of  FDTE  night  operations  was  flawed  in 
that  crews  were  allowed  to  make  excessive  use  of  light  sticks  to 
meet  task  illumination  requirements.  This  brings  into  (juestion 
the  validity  of  the  night  operations  crew  drill  performance 
findings  and  the  adequacy  of  NLOS  design  for  night  operations. 

The  NLOS  Embedded  Trainer  evaluation  originally  scheduled 
for  FDTE  was  not  conducted  due  to  the  late  arrival  of  the 
training  device.  It  was  originally  planned  to  investigate 
whether  the  Embedded  Trainer  could  be  used  to  teach  crewmen  the 
skills  required  to  guide  an  NLOS  missile  to  target  and  to  operate 
the  fire  unit.  The  2U3sence  of  training  devices  severely  limited 
the  ability  to  Investigate  NLOS  engagements  during  FDTE.  A  key 
MANPRINT  issue  that  was  identified,  but  which  could  not  be 
evaluated,  was  the  ability  of  the  NLOS  system  to  detect  moving 
targets  given  current  command  and  control  system  target  cuing 
time  delays.  The  issue  was  noted  in  the  FDTE  Test  Report  for 
later  evaluation. 
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MANPRINT  In  Extended  User  Employment 


EVE  QYgrvigv 

During  the  period  from  January  through  September  1990,  the 
API  Fort  Bliss  Field  Unit  conducted  evaluations  in  support  of  the 
NL3S  Extended  User  Employment  (EUE)  program.  The  ARI  Crew 
Weapons  Performance  Team  facilitated  the  integration  of  MANPRINT 
into  the  NLOS  EUE  progreun  by  developing  and  applying  concepts, 
tools,  and  methods  to  analyze  several  specific  MANPRINT  issues. 
The  Air  Defense  Artillery  (ADA)  School  EUE  evaluation  plan  was  to 
use  the  prototype  NLOS  fire  units  to  refine  air  defense  doctrine, 
tactics,  techniques,  and  procedures.  EUE  would  also  provide  a 
source  of  technical  and  operational  data.  Other  agencies  had  an 
opportunity  to  explore  concepts  through  their  own  evaluations 
using  the  NLOS  platoon  during  EUE. 


MANPRINT  Concerns 

NLOS  EUE  efforts  were  directed  at  developing  concepts, 
tools,  and  methods  for  low  cost  "soldier  in-the-'loop" 
evaluations.  The  evaluations  provided  decision  makers  with 
Information  on  key  concerns  in  a  timely  manner  to  support  the 
research  and  development  program  decision  schedule. 

Specifically,  NLOS  EUE  support  was  directed  toward  a  follow-up 
evaluation  of  lessons  learned  from  NLOS  FDTE  and  toward  an 
investigation  of  proposed  system  design  changes.  Specific  issues 
Identified  for  evaluation  were  as  follows: 

(1)  Gunner's  Console  Objective  System  Design, 

(2)  NLOS  Fire  Unit  Emergency  Egress, 

(3)  Prototype  System  Embedded  Trainer  Characteristics, 

(4)  NLOS  Task  Illumination, 

(5)  Vehicle  Light  Signature  Evaluation. 


Method  Overview 

The  scope  of  each  evaluation  was  restricted  to  a  limited  set 
of  design  features  or  a  single  crew  drill  so  that  evaluations 
could  be  conducted  with  available  resources,  typically  during 
normal  duty  hours,  and  as  soldier  duty  schedules  permitted. 
Videotape  recordings  were  used  as  a  primary  means  of  data 
collection,  minimizing  the  need  for  test  support  personnel. 
Intixpensive  equipment  mock-ups  were  used  as  a  tool  for  gathering 
early  soldier  input  on  proposed  design  changes.  "Motor  Pool"  and 
one-day  local  training  area  exercises  were  used  as  a  quick  and 
inexpensive  method  for  gathering  information  which  did  require  an 
operational  environment.  The  key  requirement  of  each  evaluation 
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was  to  approximate  tactical  realism  on  only  those  dimensions 
relevant  to  the  evaluation  (i.e.,  focus  on  crew  egress  tasks,  or 
capture  performance  data  on  the  embedded  trainer) . 


Evaluation  It  Gunner's  Console  Design 

Concerns.  During  NLOS  FDTE  it  was  observed  that  the  crew 
compartment  space  was  very  limited.  A  contractor  proposed 
Gunner's  Console  redesign  configuration,  presented  as  an 
engineering  drawing,  raised  the  concern  that  crew  space  might  be 
further  reduced,  thus  raising  additional  safety  concerns. 

Method.  ARI  personnel  built  mock-ups  of  the  proposed 
equipment  design  so  that  an  early  "soldier-in-the-loop"  hands-on 
evaluation  by  NLOS  gunners  might  be  conducted.  A  sheet  metal  and 
plastic  mock-up  was  fabricated  by  Fort  Bliss  engineers  for  use  in 
the  static  evaluation  of  the  design.  For  field  testing,  a 
styrofoam  and  cardboard  mock-up  of  the  gunner's  console  was  used, 
which  was  assembled  in  only  a  few  hoiirs.  Figure  4  presents  an 
Illustration  of  the  prototype  system  Gunner's  Console  design  used 
in  FDTE.  An  illustration  of  the  proposed  objective  system  design 
appears  as  Figure  5.  A  photo  of  the  ARI  produced  objective 
system  Gunner's  Console  mock-up  used  in  the  evaluations  appears 
as  Figure  6.  On  16  and  19  February,  1990,  NLOS  giinners 
participated  in  a  review  of  the  physical  design  characteristics 
of  the  proposed  Gunner's  Console  using  mock-ups  of  the  equipment 
in  a  HHMWV.  The  evaluation  included  a  static  motor  pool  review 
and  also  mobile  field  trials  at  a  local  training  area. 

Results.  The  evaluation  generated  a  large  number  of 
detailed  comments  from  the  crewmembers  regarding  features  of  the 
proposed  design  and  specific  recommendations  for  changes.  The 
Formal  Memorandum:  NLOS  EUE  Gunner  Console  Design  Mock-Up 
Evaluation,  dated  23  February,  1990,  contains  the  crewmember 
comments  and  is  Included  as  Appendix  1.  Briefly,  gxinners  stated 
that  the  proposed  console  design  blocked  the  forward  vision  of 
the  gunner's  position  and  partially  blocked  the  side  vision  of 
the  driver.  The  short  distance  between  the  console  screen  and 
gunner's  face  was  seen  as  a  safety  hazard,  though  gunners  did  not 
wish  to  have  the  console  located  under  the  dash  as  in  earlier 
system  configurations.  These  results  were  presented  at  an  NLOS 
Controls  and  Displays  Design  Review,  and  at  the  NLOS  Human 
Engineering  Working  Group  (HEWG)  Meetings.  The  HEW6  adopted  the 
identified  concerns  as  issues  for  future  evaluation. 

MANPRINT  concepts,  methods,  tools.  The  MANPRINT  concept  of 
focusing  on  soldier  issues  as  early  as  possible  in  new  system 
acquisition  is  in  reaction  to  the  decreasing  flexibility  of 
equipment  design  over  time  as  availeUale  design  engineering 
dollars  are  spent.  Users  have  a  responsibility  to  begin 
exploring  the  "soldier-equipment  fit"  issue  as  soon  as  early 
design  concepts  are  identified  to  influence  the  design  of 
equipment  to  meet  their  o%m  needs.  Equipment  mock-ups  proved  to 
be  valuable  tools  which  go  well  beyond  line  drawings  in 
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Figure  4.  Initial  system  gunner’s  console  design. 
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Figure  5.  Objective  system  gunner’s  consoie  design 
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Figure  6.  ARI  objective  system  gunner's  console 
mock-up. 
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elici'ting  early  user  Inpu't  on  proposed  designs.  Many  who  had 
taken  a  "wait  and  see"  position  about  the  new  design  had 
immediate  insights,  and  could  suggest  improvements  to  enhance  the 
"soldier-machine  fit"  of  the  system  after  having  the  opportunity 
to  experience  the  mock-up  of  the  total  workspace  environment.  By 
the  time  of  Critical  Design  Review  (CDR)  the  proposed  NLOS  design 
was  mature,  and  user  input  was  sought  for  this  specific  design, 
rather  than  for  proposing  alternative  design  elements  or 
approaches  to  design  requirements.  Valuable  user  insights  can  be 
gathered  through  such  techniques  as  the  use  of  mock-ups,  but  if 
user  input  is  to  have  an  impact  on  system  design,  this  must  occur 
very  early  in  the  design  phase,  before  engineering  dollars  are 
spent  and  design  flexibility  is  gone. 

Evaluation  2; _ Emergency  Egress 

Concerns .  Changes  proposed  for  the  NLOS  objective  system 
included  the  addition  or  redesign  of  equipment  within  the  crew 
cab  which  would  significantly  reduce  available  space.  The 
concern  was  raised  that  this  equipment  might  block  the  escape  of 
a  crewmember  from  the  crewcab  if  the  vehicle  doors  were  blocked 
in  an  accident.  No  egress  requirement  was  stated  in  the  NLOS 
system  specifications,  perhaps  because  safe  egress  is  assumed  for 
systems  based  on  the  standard  HMMWV  vehicle.  The  Crew  Weapons 
Performance  Team  chose  to  exaualne  the  impact  of  adding  the  new 
equipment  to  the  existing  HMMWV  operator  workspace. 

Method.  An  evaluation  was  conducted  to  identify  how  well 
soldiers  could  climb  over  the  equipment  separating  the  two  cab 
seats  to  exit  the  vehicle.  On  24  May,  1990,  sixty-four  emergency 
egress  trials  were  videotaped  and  timed  where  both  crewmen  were 
required  to  exit  through  the  same  NLOS  vehicle  door.  Major 
pieces  of  NLOS  equipment  (air  plenum,  radios,  gunner's  console) 
were  built  as  mock-ups  and  placed  in  the  crew  cab.  Videotape 
recordings  were  made  of  each  trial  to  aid  in  identifying  specific 
features  of  the  system  design  that  impacted  egress  performance, 
and  to  create  a  flexible  datediase  for  post  hoc  analyses.  The 
evaluation  captured  the  time  required  for  both  crewmembers  to 
egress  out  the  driver's  door,  and  the  gxinner's  door,  under 
conditions  of  MOPPO  (no  chemical  protective  equipment)  and  M0PP4 
(full  chemical  protective  oversuit  and  mask). 

Results .  The  Formal  Memorandum:  NLOS  7JUE  Crewmember  Egress 
Evaluation,  dated  30  August,  1990,  details  the  results  of  this 
evaluation,  and  is  included  as  Appendix  2.  The  memorandum 
Identifies  the  design  features  which  aided  crewmembers,  and  those 
which  impended  their  progress  in  exiting  the  vehicle,  and  details 
trials  where  crewmembers  bec2uae  entangled  in  equipment  and  were 
uned>le  to  leave  the  vehicle.  Egress  was  shown  to  be  a  difficult 
but  achieveUsle  task.  An  inexperienced  gunner  needed  as  much  as 
60  seconds  to  exit  the  vehicle,  while  the  seune  gunner,  with 
experience,  could  exit  the  vehicle  in  an  average  of  about  15 
seconds.  Performance  was  thus  very  dependent  on  experience  or 
training.  An  analysis  of  variance  test  confirmed  that  there  is 
no  significant  difference  between  the  egress  times  for  the 
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conditions  of  MOPPO  (13.73)  and  MOPP4  (13.73)  [£(1,  57),  »  0.00, 
£>0.9997]  for  the  59  trials  where  soldiers  successfully  egressed 
from  the  vehicle.  Findings  were  briefed  at  the  4  June  NLOS  HEWG 
meeting  at  the  Boeing  (Huntsville,  AL. )  facility  using  the 
videotaped  performance  trial  recordings  to  support  the  siimmary 
data.  The  egress  concern  was  formally  adopted  as  a  HEWG  design 
issue.  The  contractor  implemented  changes  in  the  objective 
system  design,  relocating  radio  equipment  out  of  the  crew  ced>, 
thereby  directly  addressing  the  egress  design  concerns. 

MANPRINT  concepts,  methods,  tools.  The  emergency  egress 
evaluation  reflected  an  emerging  user  level  concern  and  was  not  a 
formal  NLOS  design  requirement.  Early  user  input  did  influence 
system  design,  as  focusing  attention  on  this  safety  concern  led 
the  contractor  to  make  adjustments  in  the  proposed  design,  moving 
electronic  equipment  from  the  crew  cab  to  the  rear  of  the  vehicle 
and  thus  increasing  crew  cah  workspace.  In  this  case  the  early 
soldier-in-the-loop  MANPRINT  concept  worked  well  to  shape  system 
design. 


Evaluation  3;  Embedded  Trainer  Characteristics 

Concerns .  The  NLOS  Embedded  Trainer  (ET)  provided  both 
"Fixed"  scenarios  which  are  entirely  preprogrammed,  and 
"Variable"  scenarios  which  allow  the  gunner  to  select  his  initial 
fire  unit  location.  The  ET  is  the  prime  source  of  sustainment 
training  for  the  NLOS  system.  The  trainer  was  not  avail2U3le  for 
FDTE  testing,  making  it  a  prime  candidate  for  user  evaluation 
during  the  follow-on  EUE  program.  The  Crew  Weapons  Performance 
Team  elected  to  evaluate  the  NLOS  ET  system  in  terms  of  eleven 
required  technical  characteristics  for  embedded  training  systems 
that  had  been  identified  by  Purifoy,  Ditzian,  and  Finley  (1989) 
to  ensure  that  ET  concerns  were  addressed  in  a  consistent  and 
comprehensive  manner.  The  eleven  required  characteristics 
include  such  areas  as  Training  Content,  Levels  and  Types  of 
Training,  Feedback  of  Performance,  and  Logistics  Burden  issues  as 
criteria.  A  video  ceunera  was  positioned  over  the  right  shoulder 
of  the  gunner  in  the  NLOS  fire  unit  to  record  both  audio  and 
video  data  for  crewmember  performance  on  twelve  simulated 
engagements. 

Results .  Videotapes  were  reviewed  to  identify  soldier 
performance  taslcs  associated  with  the  NLOS  prototype  Embedded 
Training  system  and  preliminary  tasks  lists  were  created.  Target 
engagement  task  performance  times  and  ranges  to  target  area  for 
twelve  trial  scenarios  were  compiled  for  preliminary  evaluation 
of  gunner  task  performance  trends,  and  as  a  first  step  toward 
Identifying  measur^d^le  performance  standards  or  criteria  for  each 
scenario.  Task  performance  time  and  range  data  for  NLOS  Target 
Engagement  Scenario  #6  is  provided  as  Ted>le  4.  The  time  and 
range  data  for  task  performance  in  T2U3le  4  shows  some  apparently 
odd  trends  for  the  tasks  of  zooming  and  slewing  the  missile 
seeker,  where  the  gunner  luinlpulates  these  controls  long  before  a 
target  could  be  present.  From  observations  and  discussions  it 
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was  found  that  gunners  would  test  these  system  functions  by 
manipulating  them  early  in  engagements  to  ensure  that  they  were 
working.  Ex2unples  of  performance  feedback  displays  which  could 
be  provided  to  the  gunner  to  enhance  training  were  developed.  An 
example  of  a  possible  feedback  display  for  a  gunner  is  provided 
as  Figure  7.  A  preliminary  MicroSAINT  soldier  performance 
computer  model  was  developed  incorporating  NLOS  ET  tasks  and  task 
performance  times.  The  evaluation  of  the  NLOS  Prototype  Embedded 
Training  system  in  terms  of  the  eleven  characteristics  is 
svimmarized  in  TeJsle  5.  Memoranda  reporting  results  of  the  ET 
evaluation  were  provided  to  the  NLOS  TRAOOC  System  Manager  (TSM) 
for  use  in  developing  NLOS  engagement  task  performance  scoring 
criteria.  The  Formal  Memorandum:  NLOS  Embedded  Trainer  MANPRINT 
Evaluation,  dated  22  February,  1991,  detailed  the  results  of  this 
evaluation,  and  is  Included  as  Appendix  3.  This  memorandum 
presents  data  for  three  trials  on  each  of  four  ET  scenarios  as 
well  as  a  more  in-depth  discussion  of  the  gunner  tasks,  the 
MicroSAINT  model,  and  detailed  comments  for  each  of  the  eleven 
required  technical  characteristics  of  embedded  training  systems. 

MANPRINT  concepts,  methods,  tools.  The  MANPRINT  evaluation 
applied  the  ARI/PM  TRADE  Embedded  Training  evaluation  scheme  to 
the  prototype  NLOS  Embedded  Trainer  to  generate  early  soldier 
performance  and  training  effectiveness  information  for  equipment 
designers  and  decision  makers.  The  eleven  characteristics  scheme 
offers  a  useful  format  for  reviewing  ET  systems  early  in  their 
development.  The  need  for  performance  feedback  and  standards  is 
essential  to  training  effectiveness.  Ex2unples  of  feedback 
displays  were  developed  from  actual  NLOS  ET  soldier  performance 
data. 


EvalMatign  4i _ NLOS  Task  illumination 

Concerns .  Light  discipline  was  not  enforced  during  the 
conduct  of  NLOS  FDTE  night  crew  drills.  This  lack  of  realism  may 
have  led  to  an  overestimation  of  NLOS  crew  performance,  i.e., 
quicker  performance  times  and  fewer  errors.  A  related  issue 
which  arose  during  discussions  with  acquisition  program  members 
was  that  alternative  light  sources  existed  which  have  a  lower 
light  signature  than  the  standard  Army  issue  flashlight.  During 
NI/)S  EUE,  a  follow-on  evaluation  of  night  crew  drill  performance 
under  conditions  of  light  discipline  was  conducted.  On  25 
September  1990,  eight  single  missile  reload  battle  drills  were 
carried  out  in  a  simulated  night  environment  in  a  maintenance 
building  at  Fort  Bliss.  A  video  ceu&era  equipped  with  a  night 
vision  device  was  used  to  record  gunner  task  performance.  The 
video  recording  was  reviewed  to  extract  task  performance  times 
when  a  standard  issue  flashlight  with  a  red  filter  was  used  and 
when  a  prototype  low  signature  Cluster  Map  Light  (CML)  flashlight 
was  used.  The  infrared  signature  of  the  red  filter  flashlight,  a 
blue-green  filter  flashlight,  and  CML  flashlight  lined  up  side  by 
side  was  recorded  on  video  tape  to  provide  a  comparison  of  their 
relative  signature  sizes. 
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Results.  The  Formal  Memorandvim:  NLOS  EUE  Task  Illumination 
MANPRINT  Evaluation,  dated  10  December  1990,  detailed  the  results 
of  this  evaluation,  and  is  included  as  Appendix  4.  The  crews 
were  able  to  perform  reload  tasks  and  displayed  similar  task 
performance  times  with  both  the  red  filter  and  CML  flashlights 
(see  Table  6) .  The  general  finding  was  that  crews  could  perform 
single  missile  reload  tasks  with  little  or  no  illumination.  In 
comparison  to  the  standard  issue  red  filter  flashlight,  the  blue- 
green  filter  reduced  the  flashlight  signature  to  £d>out  60%,  while 
the  prototype  CML  flashlight  signature  was  only  about  11%  of  that 
of  the  red  filter  flashlight.  Giinners  recommended  adding  light 
source  brackets  in  the  crew  cad)  area  on  the  radio  rack  to  allow 
for  lap  level  illumination.  There  was  no  recommendation  for 
adding  light  source  brackets  on  exterior  points  of  the  vehicle  to 
aid  missile  reload  task  performance.  These  findings  were 
presented  at  a  30  October  1990,  NLOS  briefing  at  Fort  Bliss. 

MANPRINT  concepts,  methods,  tools.  The  task  illumination 
evaluation  showed  that  night  operations  could  be  conducted  in 
equipment  bays  to  address  equipment  design  Issues.  This  one-day, 
no  cost,  motor  pool  evaluation  allowed  for  more  repetitions  of 
the  single  missile  reload  night  crew  drill  than  had  occurred  in 
the  previous  four  weeks  of  field  testing.  The  night  vision  video 
recordings  of  soldier  performance  were  extremely  useful  in 
minimizing  the  h\iman  data  collector  requirement  and  in  allowing 
for  detailed  post-trial  time  analysis. 


Evaluation  5; _ NI/3S  Vehicle  Light  Signature 

Concerns .  The  experience  of  watching  NLOS  crews  perform 
crew  drills  at  night  raised  the  concern  that  the  light  signature 
given  off  by  the  vehicle  electronics  and  crew  flashlights  would 
make  the  system  vulnerable  to  detection  and  attack  in  an 
operational  environment.  The  Crew  Weapons  Performance  Team 
raised  this  concern  at  an  NLOS  Human  Engineering  Working  Group 
meeting  and  agreed  to  pursue  the  question  in  more  detail.  The 
task  was  to  identify  a  low  cost  method  and  criteria  for  measuring 
vehicle  night  illumination  signature  that  could  be  used  early  in 
system  development,  and  to  apply  this  method  to  the  prototype 
NLOS  system  so  as  to  establish  a  preliminary  night  signature 
baseline.  The  Army  Human  Engineering  Lab  (HEL)  provided  the 
criterion  that  light  sources  should  not  add  to  vehicle 
detect2U3llity  at  a  distance  of  50  meters.  Subjective  estimates 
of  light  source  signature  size  were  collected  by  three 
independent  observers.  Two  used  PNV-4  night  vision  goggles.  One 
observer  did  not  use  any  vision  aids.  Light  signature  estimates 
were  made  from  four  locations  in  a  50  meter  circle  extending  from 
the  vehicle  to  provide  a  front,  right,  rear,  and  left  side  view. 
Five  light  sources  were  evaluated,  the  white  light  coming  from 
the  NLOS  Gxinner's  Console,  red  filter  and  blue-green  filter 
flashlights,  a  low  signature  cluster  flashlight,  and  a  low 
signature  cluster  map  light.  Data  collection  took  place  on  the 
night  of  27  September  in  a  local  training  area  at  Fort  Bliss. 


Results.  Results  of  the  light  source  signat\ire  evaluation 
are  summarized  in  T^d}le  7.  Gunners  stated  that  they  had  adequate 
Illumination  for  task  performance  with  each  of  the  light  sources. 
Night  task  performance  using  the  ciirrent  Army  issue  red  filter 
flashlight  produced  a  vehicle  signature  detectable  to  night 
vision  goggles  and  the  unaided  eye  at  a  distance  of  50  meters 
from  the  vehicle,  when  observed  from  the  four  viewing  positions 
separated  by  90  degrees.  The  blue-green  flashlight  filter,  and 
the  CHL  flashlight  and  cab  light  illumination  sources  yielded  a 
vehicle  signature  that  was  very  difficult  to  detect  with  night 
vision  goggles  or  the  unaided  eye  from  any  point  at  a  50  meter 
radius  from  the  vehicle.  These  findings  were  presented  at  the  30 
October  1990,  NLOS  Laydown  briefing  at  Ft  Bliss,  Texas. 

MANPRINT  concepts,  methods,  tools.  The  vehicle  light 
signature  evaluation  resulted  in  a  simple  procedure  and  set  of 
performance  criteria  that  can  be  used  in  a  one-day  evaluation  of 
any  system.  The  thrust  of  the  effort  was  to  validate  and 
quantify  the  HANPRINT  concern  regarding  NLOS  night  signature,  and 
to  provide  an  exzunple  to  the  NLOS  acquisition  community  of  how 
low  cost  evaluations  could  be  conducted  early  on  in  system 
development.  The  results  suggest  that  the  low  signature  light 
sources  might  be  a  useful  alternative  to  the  current  flashlights. 


Conclusions: _ NI/)S  EUE  Evaluation 

The  NLOS  Extended  User  Employment  MANPRINT  efforts  were 
directed  at  developing  and  applying  concepts,  methods,  and  tools 
to  analyze  NLOS  system  concerns.  During  NLOS  EUE,  the 
opport\inlty  existed  to  conduct  low  cost  "soldier  in-the-loop" 
evaluations  to  generate  user  data  and  to  provide  decision  makers 
with  information  on  key  concerns.  Several  lessons  learned  can  be 
identified  from  the  effort.  An  EUE  progreua  can  provide  a 
valucible  opportunity  for  follow-up  evaluations  based  on  lessons 
learned  from  the  broader  testing  program.  Earlier  is  better  when 
investigating  "man  in  the  loop"  concerns  because  design 
flexibility  decreases  rapidly  after  a  design  concept  is 
identified.  Evaluations  of  limited  scope  can  be  conducted 
quickly  and  inexpensively  in  the  user  motor  pool  or  local 
training  area  with  commonly  available  resources.  Such 
evaluations  can  provide  a  valuable  "quick  look"  at  user  concerns 
to  determine  whether  these  need  to  be  raised  as  formal  issues 
within  the  system  research  and  development  progreun.  Simple  mock- 
ups  of  proposed  equipment  can  generate  valuable  insights  that  can 
Influence  system  design  when  identified  early.  The  use  of  video 
tape  recordings  of  crew  and  system  performance  is  a  flexible  and 
low  cost  method  which  supports  both  data  collection  and  analysis. 
Finally,  the  use  of  video  taped  crew  performance  is  a  powerful 
means  of  conveying  safety  concerns  to  system  developers. 
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Table  4 

Embedded  Trainer  Target  Engagement  Task  Performance 


Trial  3 

*Time/**Range 

000/5150 

008/4500 

018/3458 

022/3086 

015/3880 

016/3500 

034/1640 

036/1481 

048/499 

109/499 

None 

Don’t  Know*** 

122/Hit 

Trial  2 

*Time/**Range 

00/5150 

00/4500 

10/4200 

12/3900 

12/3900 

13/3800 

18/3310 

40/1200 

None 

None 

None 

Don’t  Know*** 

48/Hit 

Trial  1 

*Tlme/**Range 

00/5150 

08/4500 

36/1480 

11/4143 

07/4560 

11/4143 

30/2229 

30/2121 

30/2067 

33/1906 

None 

Don’t  Know*** 

48/Hit 

Engagement  Tasks 

Enable  Fire 

Auto  Search  Start 

Zoom  Seeker  Manually 

Slew  Seeker  Manually 

Target  In  FOV 

Target  Detected 

End  Seeker  Slew 

First  Lock-On  Try 

First  Break-Lock 

Second  Lock-On  Trv 

Second  Break-Lock 

Target  Identified 

Missile  Hits  Target 
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Cumulayve  engagement  time  in  seconds  from  launch/range  to  original  target  area  in  meters. 
*  Decreasing  range  to  target  area  stops  arbitrarily  at  499  meters. 

**  No  required  action  associated  with  Target  Identification  task. 
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Figure  7.  Gunner  performance  feedback:  Comparison  to  "window' 
standard. 


Table  5 

NLOS  Embedded  Trainer  Characteristics  Evaluation 


1.  TIME  REQUIRED  TO  TRANSITION  INTO  AND  OUT  OF  ET. 

Gunners  transition  into  and  out  of  ET  in  10-15  seconds. 

2.  ET  LOGISTICS  BURDEN. 

No  ET  logistics  concerns  were  identified. 

3.  LEVELS  AND  TYPES  OF  TRAINING  DELIVERED  BY  ET. 

Concern;  Content  is  limited  to  target  engagement, 
could  be  broadened  in  scope  to  optimize  total  POI  ET 
sustainment  training  potential. 

4.  NUMBER  AND  TYPES  OF  TRAINING  DELIVERED  BY  ET. 

"Variable”  scenarios  provide  flexibility  needed  to 
prevent  memorization. 

5.  TRAINING  CONTENT  OR  TOPICS. 

Concern;  The  prototype  ET  system  automatically  provides 
a  target  intercept  point  to  the  gunner.  This  artificial 
feature  eliminates  key  mental  task  training. 

6.  NEED  FOR  INSTRUCTORS  OR  TRAINING  MANAGERS  WHEN  USING  ET 
Gunners  exercise  ET  without  assistance,  replay  with  on¬ 
board  videotape. 

7.  REQUIRED  LEVEL  OF  FIDELITY  OF  SIMULATIONS. 

Constraints;  Single  missile,  daylight  only,  no  target 
identification,  no  time  pressure  to  launch,  automatic 
target  intercept  calculation. 

8.  PERFORMANCE  EVALUATION  AND  ASSESSMENT  CAPABILITY. 
Concern;  No  performance  summary  or  record  except  video¬ 
tape.  No  diagnostic  comparison  of  task  performance  to 
standards. 

9.  PERFORMANCE  DATA  SECURITY. 

The  only  potential  data  source  is  the  optional  videotape 
record. 

10.  FEEDBACK  IN  SUPPORT  OF  EFFECTIVE  LEARNING. 

Concern;  No  performance  feedback  against  a  standard. 

11.  ET  ADAPTABILITY,  RESPOND  TO  SOLDIER  PERFORMANCE  AND 
STREGHTEN  WEAK  AREAS. 

Concern;  Need  to  evaluate  performance  and  provide  a 
software  message  to  direct  gunner  to  appropriate  ET 
scenario  or  lesson  in  a  training  matrix. 
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Table  6 

Night  Missile  Reload  Times  By  Crew  and  Light  Source 


\.Source 

Red  Filter 

Cluster  Light 

Red  &  Cluster 

Crew 

Crew  1 

7  min  40  sec 

6  min  12  sec 

6  min  56  sec 

Crew  2 

8  min  22  sec 

8  min  46  sec 

8  min  34  sec 

Crew  1  &  2 

8  min  01  sec 

7  min  29  sec 

7  min  45  sec 
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Table  7 

Light  Source  Visibility  by  Night  Vision  Goggles  (NVG)  and  Unaided  Eye 
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Red  Filter  light  source  visible  through  vehicle  right  mirror  reflection. 


Estimating  NLOS  Target  Search  Demsuids 
Command  and  Control  Cuina  Delays  and  Target  Acquisition 

Total  system  perspective;  Command  and  control  delays.  NIOS 
typically  depends  on  an  independent  sensor  or  forward  observer  to 
provide  the  target  information  needed  for  engagements.  This 
introduces  delays  ranging  from  a  few  seconds  to  several  minutes 
before  the  information  arrives  at  the  NLOS.  The  time  reguired  to 
send  target  information  to  the  NLOS,  and  for  the  NLOS  to  send  a 
missile  to  the  target  area,  provides  an  opportunity  for  the 
target  to  move  from  its  original  location.  As  engagement  time 
increases  the  potential  area  in  which  a  target  could  move  also 
increases,  while  the  proportion  of  this  target  area  that  can  be 
covered  by  the  missile's  field  of  view  decreases.  A  major 
question  surrounding  the  fielding  of  NLOS  is  whether  the  system 
will  be  able  to  find  targets  using  current  command,  control,  and 
intelligence  (C2I)  systems,  or  whether  NLOS  is  dependent  upon  a 
future  C2I  system  to  provide  near  realtime  transmission  of  target 
information  and  a  target  location  update.  The  NLOS  target  search 
capedaility  was  a  key  MANPRINT  concern.  As  longer  target  cuing 
delay  times  allow  for  greater  target  movement,  successful  target 
acquisition  should  depend  increasingly  on  the  skills  and 
ad)illties  of  the  gimner  to  rapidly  integrate  information  and 
predict  a  target  Intercept  location. 

To  address  the  question  of  NLOS  target  search  effectiveness 
given  C2I  constraints,  the  Crew  Weapons  Performance  Teeun 
developed  a  series  of  simple  distance  and  area  calculations  to 
estimate  the  proportion  of  the  potential  target  movement  area 
that  NLOS  can  search  as  a  function  of  target  cuing  delays. 
Engagement  times  corresponding  to  a  total  of  six  current  and 
future  command  and  control  situations  were  used  for  the 
estimates.  The  logic  here  is  that  with  longer  cuing  delays  the 
potential  target  movement  area  expands  rapidly,  while  the 
proportion  of  the  expanding  target  movement  area  that  can  be 
searched  by  NLOS  decreases.  The  team  used  the  paper  and  pencil 
search  area  estimates  to  raise  the  issue  of  target  movement  and 
NLOS  search  area  coverage  to  Army  Defense  Simulation  Network- 
Developmental  (SIMNET-D)  testers.  The  target  movement  issue  was 
formally  adopted  for  evaluation  in  the  NLOS  SIMNET-Developmental 
test.  Phase  I,  which  was  conducted  1  through  17  April  1991,  at  Ft 
Rucker ,  A1 abama . 

Estimating  target  movement  during  NIX)S  engagements.  NLOS 
capedsility  to  find  rotary  wing  targets  may  be  limited  due  the 
slow  speed  of  the  missile  compared  to  target  speeds  and  the  time 
reguired  to  convey  target  Information,  launch  the  missile,  and 
fly  to  the  target  area.  The  NLOS  missile  travels  at  a  relatively 
low  speed  of  approximately  100  meters  per  second  (a  200  meters 
per  second  design  has  been  explored) ,  compared  to  a  top  speed  of 
320km/hr  or  88  meters  per  second  for  the  standard  threat  attack 
helicopter,  the  Ni-24  Hind.  The  Ft  Bliss  Directorate  of  Combat 
Developments  estimated  that  target  information  sent  through  six 
current  and  future  command  and  control  systems  would  take  from  0 
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seconds  (NCTR  sensor)  to  160  seconds  (Combined  Arms  [CA]  Voice 
Net)  to  reach  the  NLOS  fire  unit  (see  Figure  8) .  Missile  launch 
would  require  30  seconds,  and  a  10km  missile  flight  to  a  target 
area  would  require  another  100  seconds,  so  that  engagement  times 
against  a  10km  dist2uit  target  would  range  from  130  to  290 
seconds.  Figure  9  shows  how  far  a  target  moving  at  5 
meters/second  (hovering) ,  40  meters/second  (half  speed) ,  and  80 
meters/second  (full  speed)  could  travel  during  the  six  engagement 
times.  For  ex£unple,  a  target  moving  at  80  meters/second  could 
travel  10.4  kilometers  during  a  130  second  (NLOS  with  on-board 
sensor)  engagement. 

Proportion  of  target  movement  area  searchable  bv  NLOS. 

Given  estiioates  of  the  distance  that  a  target  can  travel  during 
an  NLOS  engagement  before  being  overtaken  by  the  missile,  one  can 
calculate  the  area  in  which  the  target  can  move  and  the 
proportion  of  this  target  area  that  can  be  covered  by  the  NLOS 
missile's  2km  wide  seeker  field  of  view.  The  proportion  of  a 
potential  target  area  that  can  be  searched  by  NLOS  was  estimated 
based  on  the  distance  a  target  can  travel  during  an  engagement, 
and  the  area  that  can  be  searched  by  the  NLOS  missile  seeker 
using  a  formula  that  takes  into  consideration  target  cuing  delay, 
missile  speed,  and  target  speed  vari2d}les.  For  purposes  of 
estimation  it  was  assumed  that  a  target  would  only  move  away 
from,  or  parallel  to,  the  NLOS  fire  unit,  so  that  the  potential 
target  movement  area  is  represented  by  a  half-circle. 

The  Percentage  of  Target  Area  Searchable  (PTAS)  by  the  NLOS 
missile  can  be  estimated  for  a  lOOm/sec  missile,  with  a  65  second 
(Manual  SHORAD  Control  System  (MSCS))  cue  delay,  30  second  launch 
time,  and  100  second  flyout  time  (195  seconds  total  untill 
missile  reaches  target  area)  against  a  10km  distant  rotary  wing 
target  moving  40m/sec  as  follows: 


Given: 

a  -  Time  for  target  cuing,  missile  launch,  and  flight  to 
target  area  (in  seconds) 

b  «  Missile  Speed  (in  meters  per  second) 

c  ■■  Target  Speed  (in  meters  per  second) 

FOV  >  Missile  seeker  Field  Of  View  (2km  wide) 


1.  Target  Distance  Traveled  (TDT)  ■  fa  X  c)  X  b 

(b  -  c) 

TDT  -  f65  +  30  100  sec^  X  40  m/sec  X  100  m/sec  «  13km 

(100  m/sec  -  40  m/sec) 
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2.  Target  Novenent  Area  (TMA)  *  TDT^  X  (3.1416  /  2) 
TMA  «  13kn^  X  (3.1416  /  2)  «  265.47)01  eq 


3.  Missile  Seelcer  Search  Area  (MSSA)  «  FOV  X  TDT 
NSSA  -  2)01  X  131cb  -  261cb  sq 


4.  Percentage  of  Target  Area  Searchable  (PTAS)  *  TMA  /  MSSA 
PTAS  -  265.47)01  sq  /  26)oi  sq  «  9.79% 


For  the  MSCS  )3ased  engagement  exemple  presented  ahove,  it 
would  ta)ce  195  seconds  from  target  cuing  initiation  until  the 
missile  reaches  the  target  coordinates  at  10)01  distance.  The 
target  would  be  able  to  move  up  to  13)oi  in  any  direction  prior  to 
being  overta)cen  by  the  missile.  The  potential  movement  area 
beyond  the  target's  initial  position  would  be  265)cm  sq,  9.79%  of 
which  could  )>e  searched  by  the  NLOS  missile  see)cer.  A  graphic 
representation  of  this  target  engagement  example  is  presented  as 
Figure  10.  The  probability  of  bringing  targets  into  the  see)cer 
field  of  view  should  rise  above  the  9.79%  estimate  based  on 
random  missile  search  after  reaching  the  original  target 
coordinates,  if  target  movement  can  be  predicted  and  a  target 
intercept  course  implemented.  The  NLOS  gimner  would  use  his 
)cnowledge  of  how  geography  and  tactics  constrain  the  movements  of 
targets  (Intelligence  Preparation  of  the  Battlefield) ,  and  any 
avail€d>le  target  information  to  anticipate  target  movement. 


SIMNET-D  Target  Movement  and  Target  Accniisition  Evaluation 

One  of  the  major  user  test  issues  for  the  NLOS  SINNET-D 
evaluation  was  provided  by  the  Crew  Weapons  Performance  Team. 

The  test  issue  examined  the  effect  of  cue  delay  on  target 
availability,  specifically,  whether  command  and  control 
information  delays  and  other  target  engagement  time  requirements 
would  allow  threat  rotary  wing  aircraft  to  leave  a  searchedsle 
target  area  before  the  NLOS  missile  could  arrive.  Previous  NLOS 
live  fire  and  simulated  engagement  testing  could  not  address  this 
target  availability  issue  )>ecause  the  cost  of  testing  made  it 
necessary  that  a  target  always  ]3e  avail2d>le  for  engagement  in  the 
target  area.  The  cost  constraint  thus  eliminated  this  important 
command  imd  control  element  from  NLOS  operational  testing. 

Through  a  series  of  briefings  addressing  command  and  control 
cuing  delay  and  target  movement  issues,  the  teeus  was  able  to  have 
this  )cey  issue  incorporated  in  the  NLOS  SIMNET-D  evaluation. 

SIMNET-D  provided  a  computer  generated  virtual  battlefield 
environment  for  NLOS  target  engagement  assessments  which  Included 
terrain  views,  friendly  and  hostile  forces,  and  individual 
weapons  systems  status  and  location  reports.  The  SIMNET-D 
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Figure  10.  Target  movement  and  percentage  of  movement  area  searchable 
by  NLOS  for  a  10km  Manual  SHORAD  Control  System  (MSCS) 
engagement. 


simulators  were  realistic  mock-ups  of  the  objective  NLOS  weapon 
system  and  were  operated  by  military  personnel  familiar  with  the 
system.  The  SIMNET-D  NLOS  fire  units  were  cued  on  rotary  wing 
and  armor  target  positions  and  were  cleared  to  launch  missiles. 
Rotary  wing  tactics  included  hovering,  pop-up,  and  transiting 
maneuvers . 

For  the  NLOS  SIMNET-D  test,  revised  cue  delay  values  were 
used  to  represent  target  cuing  delays  for  automatic  and  manual 
systems.  Rotary  wing  targets  were  presented  to  gunners  with  cue 
delays  of  4,  15,  40  seconds  (automatic  cuing),  and  15,  40,  85, 
140,  190  seconds  (manual  cuing).  Gunners  were  given  target  type, 
speed,  and  direction  of  travel  which  allowed  them  to  roughly 
estimate  a  missile  flight  course  to  follow  to  intercept  the 
targets.  The  SIMNET-D  report  (Wyant,  1991)  presents  results 
indicating  whether  targets  were  available  for  detection  and 
engagement  at  the  end  of  missile  flyout  for  each  of  the  six  cue 
delay  conditions.  Unfortxinately,  the  SIMNET-D  Test  Report  does 
not  present  target  availedaillty  results  as  a  ftmction  of  target 
speed  and  distance,  which  would  be  needed  for  a  direct  comparison 
to  a  calculated  PTAS  value.  SIMNET-D  results  were  that,  overall, 
rotary  wing  targets  were  avall2d>le  to  gunners  on  377  of  640 
trials  (59%) ,  so  that  on  over  40%  of  all  trials  the  target  had 
left  the  search  area  of  the  missile  prior  to  its  arrival.  These 
results  confirm  the  importance  of  the  target  cuing  delay  and 
target  movement  issues. 

A  more  detailed  account  of  the  impact  of  specific  cuing 
delay  times  on  SIMNET  rotary  wing  target  availability  is 
presented  in  Figure  11  (Wyant,  1991) .  As  would  be  expected, 
lengthening  the  cue  delay  decreased  the  probability  that  a  target 
was  avail2U3le  when  the  missile  reached  the  cued  target  area. 

Given  a  future  FAAD  C2I  system  caped>le  of  conveying  a  cuing 
message  to  the  correct  NLOS  Fire  Unit  with  only  a  4  second  delay, 
SIMNET-D  results  Indicate  that  there  was  a  95%  probability  that 
the  target  would  be  availedile  in  the  missile  seeker  field  of  view 
when  it  arrived  at  the  target  area.  For  engagements  conducted 
with  the  longer  85  second  cue  delay,  more  typical  of  the  Army's 
current  Manual  SHORAD  Control  System  (MSCS) ,  targets  fell  within 
the  missile  search  path  on  50%  of  the  trials.  A  longer  cuing 
delay  of  190  seconds,  associated  with  the  proposed  use  of  the 
Combined  Arms  communications  network,  left  targets  within  the 
missile  search  path  on  only  20%  of  the  trials. 


Conclusions;  Target  Culna  Delays  and  NIPS  Target  Acquisition 

One  ARI  contribution  to  NLOS  research  and  development  has 
been  the  identification  of  the  command  and  control  target  cuing 
delay  as  a  key  factor  limiting  NLOS  target  engagement 
capzUslllties.  The  MANPRINT  evaluation  of  this  issue  yielded  a 
quantitative  estimate  of  the  impact  of  command  and  control  delays 
on  one  aspect  of  NLOS  target  acquisition  performance.  The 
potential  impact  of  target  cuing  delays  was  initially  identified 
from  a  simple  paper  and  pencil  analysis.  The  formula  developed 
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Cue  Delay  (in  seconds) 

Figure  11.  SIMNET-D  command  and  control  cue  delays  and  target 
availability. 


by  ARI  provided  a  quantitative  estimate  of  the  missile's  target 
area  search  capability  for  six  command  and  control  options.  The 
formula  simply  estimates  the  proportion  of  the  potential  target 
movement  area  that  can  be  covered  by  the  NLOS  seeker,  and  thus 
reflects  the  likelihood  of  finding  a  target  within  its  potential 
movement  area  by  chance.  The  command  and  control  delay  issue  was 
Integrated  into  the  SIMNET-D  testing  in  order  to  validate  the 
concern  from  within  the  formal  NLOS  acquisition  program. 

The  extent  to  which  the  NLOS  seeker  can  cover  a  target  area 
is  a  key  MANPRINT  concern  because  as  the  system  caped>illty  is 
reduced  by  cuing  delays,  the  task  of  finding  targets  should 
depend  increasingly  on  the  skills  and  abilities  of  the  Gunner. 
Knowledge  of  target  direction  of  travel  and  speed  should  enable 
an  NLOS  Gunner  to  effectively  reduce  the  potential  target 
movement  area  and  Increase  the  likelihood  of  NLOS  target 
acquisition  above  the  "chance"  target  acquisition  estimate.  The 
paper  and  pencil  target  acquisition  estimates  produced  in  this 
effort  provide  one  source  of  quantitative  Information  that  can 
serve  as  a  basis  for  estimating  how  great  a  demand  will  be  placed 
on  Gunner  skills  to  predict  target  movement  and  intercept 
coordinates  for  successful  target  engagements  for  six  command  and 
control  options. 


Conclusions 


The  goal  of  the  NLOS  MANPRINT  support  effort  was  to  develop 
and  apply  concepts,  methods,  and  tools  for  the  evaluation  of 
MANPRINT  concerns  to  support  the  research  and  development  program 
of  the  NLOS  system.  The  broad  objectives  of  this  effort  were 
threefold: 

1.  Provide  or  develop  user-level  methods  for  human  factors 
evaluation,  from  source  selection  through  fielding. 

2.  Develop  effective  strategies  for  integrating  MANPRINT 
information  and  requirements  into  the  weapon  procurement  process. 

3.  Demonstrate  MANPRINT  potential  for  improvements  to  force 
readiness,  system  operational  availability,  fire  unit  prob2d}ility 
of  kill,  weapon  system  design,  and  the  optimization  of  manpower, 
personnel,  and  training  resources. 


UseUsility  research  from  the  field  of  human-computer 
Interface  design  has  produced  design  evaluation  guidelines  and 
recommendations  that  were  useful  in  deciding  on  an  approach  to 
the  NLOS  MANPRINT  evaluation.  Booth  and  Marshall  (1989) 
identified  the  utility  of  collecting  qualitative  data  in  the 
design  development  stage  that  could  suggest  design  changes  to 
developers,  while  Whiteside,  Bennett,  and  Holtzblatt  (1988)  and 
Landauer  (1991)  stress  the  need  to  examine  system  performance 
from  the  user  perspective  in  a  realistic  task  environment. 
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Carroll  and  Rosson  (1985)  argue  for  the  use  of  nock-ups  of 
equipment  early  in  development  to  facilitate  the  generation  of 
design  ideas,  while  Virzi  (1992)  has  found  that  even  small 
numbers  of  representative  users  will  be  able  to  identify  the 
majority  of  design  problems  from  early  hands-on  design  reviews. 

A  number  of  MANPRINT  issue  evaluations  were  conducted  for 
the  NLOS  pre-production  system  by  the  Army  Research  Institute 
(ARI)  Fort  Bliss  Crew  Weapons  Performance  Team,  within  the  NLOS 
Force  Development  Test  and  Experimentation  (FDTE)  evaluation,  the 
Performance  Teeun  excunined  soldier  task  performance  and  surveyed 
hvman  factors  issues  of  system  design.  For  the  Extended  User 
Employment  (EUE) ,  the  Performance  Teeun  carried  out  a  series  of 
low  cost  evaluations  using  the  pre-production  system  and 
equipment  mock-ups.  For  the  NLOS  Defense  Simulation  Network- 
Developmental  (SIMNET-D)  evaluation,  the  Task  Force  used  a  paper 
and  pencil  analysis  to  raise  the  issue  of  NLOS  engagement 
effectiveness  given  target  cuing  time  delays  inherent  in  the 
required  command  and  control  system. 

Results  from  MANPRINT  evaluations  support  each  of  the  three 
major  objectives: 


Objective  1:  Provide  or  develop  user-level  methods  for 
MANPRINT  evaluation  from  source  selection  through  fielding.  The 
NLOS  FDTE  MANPRINT  effort  refined  the  MANPRINT  performance  error 
coding  scheme,  narrowing  the  scope  of  error  scoring  to  enhance 
reliability  while  still  meeting  the  Information  needs  of  the 
test-fix-test  program.  A  human  factors  evaluation  framework 
directly  linked  to  the  MIL  STD  1472  Human  Factors  Engineering 
(HFE)  Guidelines  was  introduced,  which  provided  a  systematic  and 
comprehensive  fr2unework  for  evaluation  that  was  used  throughout 
the  follow-on  NLOS  evaluations.  The  issue  of  operator  workload 
was  evaluated  using  a  simple  field  data  collection  method.  EUE 
was  conducted  as  a  series  of  separate  low  cost  follow-up 
evaluations  of  lessons  learned  from  NLOS  FDTE  and  of  proposed 
design  changes.  EUE  MANPRINT  evaluations  supported  decision 
maker  information  needs  prior  to  the  NLOS  Critical  Design  Review. 
Issues  identified  for  evaluation  were  the  Gunner's  Console 
design,  crew  emergency  egress,  the  Embedded  Trainer,  crew  drill 
night  performance,  and  the  vehicle  light  signature. 


Objective  2:  Develop  effective  strategies  for  Integrating 
MANPRINT  information  and  requirements  into  the  weapons 
procurement  process.  ARI  chaired  the  Manpower,  Personnel,  and 
Training  Sub-Group  of  the  NLOS  System  MANPRINT  Joint  Working 
Group  (MJWG) ,  Integrating  SMMP  issues  with  the  NLOS  FDTE  test 
plans.  ARI  assisted  in  preparing  and  executing  FDTE  test  plans, 
participated  in  all  aspects  of  field  data  collection  and  data 
analysis,  and  co-authored  the  final  NLOS  FDTE  Test  Report.  ARI 
personnel  integrated  MANPRINT  information  and  requirements  into 
the  NLOS  weapons  procurement  process  by  serving  on  the  NLOS 
Source  Selection  and  Evaluation  Board.  The  results  of  the 
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several  EUE  MANPRINT  evaluations  were  distributed  as  formal 
memoranda  through  the  NLOS  TRAOOC  System  Manager  to  the  NLOS 
Progreun  M2mager,  the  NLOS  Human  Engineering  Working  Group,  the 
NLOS  System  Safety  Working  Group,  equipment  developers,  and  other 
agencies  involved  in  NLOS  development. 


Qb^ective  3;  Demonstrate  MANPRINT  potential  for 
Improvements  to  force  readiness,  system  operational  avail2Q7ility, 
fire  unit  probedsility  of  kill,  and  manpower,  personnel,  training, 
and  weapons  system  design.  The  MANPRINT  effort  was  directed  in 
part  toward  identifying  both  the  nature  and  the  cause  of  human 
performance  errors,  so  that  remedies  might  be  found  to  reduce 
these  errors.  The  potential  for  MANPRINT  contribution  to 
improved  force  readiness  through  error  assessment  can  be 
estimated  through  some  simple  calculations.  As  example,  in  NLOS 
FDTE  a  six  percent  error  rate  for  Launch  Control  tasks  was 
Identified.  Program  plans  called  for  the  acquisition  of  16,050 
missiles  at  a  cost  of  roughly  $50,000  each.  If  Launch  Control 
errors  lead  to  mission  failure  then  the  six  percent  figure  could 
translate  into  a  wastage  of  963  missiles,  at  a  cost  of 
$48,150,000.  If  these  Launch  Control  errors  could  be  reduced  by 
50%  through  training  or  redesign,  a  cost  savings  of  $24,075,000 
might  be  realized. 

The  EUE  MANPRINT  effort  demonstrated  the  ability  of  the 
MANPRINT  approach  to  influence  the  design  of  new  systems. 

Concerns  regarding  the  overcrowding  of  the  crew  compartment  and 
related  safety  issues  led  to  the  relocation  of  some  equipment 
items  to  enhance  operator  safety.  The  NLOS  SIMNET-D  effort 
demonstrated  the  ability  of  the  MANPRINT  progreun  to  isolate  and 
address  key  issues  of  system  operational  effectiveness.  The 
simple  NLOS  target  movement  and  search  area  estimates  were  of 
sufficient  power  to  raise  the  target  search  issue  to  the  level  of 
a  formal  test  issue  for  the  NLOS  SIMNET-D  evaluation.  The 
concern  that  target  cueing  delays  inherent  in  the  required  target 
cuing  command  and  control  systems  could  greatly  reduce  NLOS 
engagement  effectiveness  was  supported  by  the  SIMNET-D  test  data 
for  target  detection. 
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APPENDIX  A 


NLOS  EUE  GUNNER  CONSOLE  DESIGN  EVALUATION 


DEPARTMENT  OF  THE  ARMY 
U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Fort  Bliss  Field  Unit,  P.O.  Box  6057 
Fort  Bliss,  Texas  79906-0057 


PERI-SB  (701n)  23  Febmaiy  1990 

MEMORANDUM  THROUGH  DIR,  ARI  FAAD  MANPRINT  TASK  FORCE  (ATTN: 
DR  DEPONTBRIAND) 

TO:  TRADOC  SYSTEM  MANAGER,  ATSA-TSM-F,  ATTN:  MAJ  FEDAKO 

SUBJECT:  NLOS  EUE  GUNNER  CONSOLE  DESIGN  MOCK-UP  EVALUATION 

1.  Purpose:  Have  NLOS  gunners  and  other  members  of  the  NLOS  community  make  an 
initial  hands-on  review  of  the  proposed  Boeing  gunner  console  design  at  least  thirty  days 
prior  to  an  anticipated  15  March  NLOS  Critical  Design  Review,  to  e:q>lore  whether  the 
gunner’s  console  physical  dimensions  and  location  might  impact  required  task 
performance  and  system  safety  requirements. 


2.  Method:  Three  mock-ups  of  the  Boeing  gunner  console  design  were  built  and  used  in 
hands-on  evaluations  Ity  gunners  on  16  and  19  February  1990  at  Ft  Bliss.  Each  mock-up 
includes  three  rectangular  boxes  measuring  12  X  5  X  8.75  inches,  8  X  11.23  X  10  inches, 
and  12  X  5  X  5  inches,  generally  meeting  the  dimensions  presented  in  the  updated 
diagram  "Gunner’s  Console  Design  Concept  (Dimensional)”  provided  as  enclosure  1.. 
Psqrer  representation  of  switches  and  buttons  were  attached  to  box  faces  consistent  with 
the  "NLOS  Guimer’s  Console"  diagram  provided  as  enclosure  2. 

a.  Sheet  metal  mock-ups  of  the  gunner  console  and  armrest  controls  were  used  for 
static  evaluatioiL  Photographs  were  taken  of  the  NLOS  workspace  with  gunners  wearing 
Load  Bearing  Equipment  (LBE)  and  MOPP  4  gear  (see  enclosure  3). 

b.  A  Styrafoam  and  cardboard  mock-up  was  used  in  road  testing  of  the  gunner 
console  design  at  Training  Area  #30  (the  dust  bowl)  at  Ft  Bliss.  This  soft  gunners 
console  was  used  to  ensure  that  gunners  would  not  be  injured  if  they  fell  against  the 
gunner  console  during  o£f-road  movement  during  hard  breaking  at  10  mph. 

c.  A  three  piece  plywood  mock-up  of  the  separate  gunner  console  conqmnents  was 
used  to  allow  gunners  to  e]q>lore  various  arrangements  of  the  three  conqmnents  in  a 
standard  HMMWV  and  an  NLOS  lOE  fire  unit. 

d.  A  four-door  "command  vehicle"  HMMWV  was  used  for  static  and  mobile  gunner 
console  design  evaluations,  as  it  provided  a  conventional  under  dash  heater/blower 
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fq^^jptn<»nt  arrangement  instead  of  the  modified  arrangement  present  in  NLOS  lOE  fire 
units.  The  "command  vehicle"  HMMWV  was  fitted  with  SINCGARS  equipment  and  a 
doth  partition  was  placed  directly  behind  the  front  two  seats  to  create  the  back  wall  and 
related  space  constraints  present  in  the  NLOS  fire  unit  design. 

e.  Soldiers  were  four  experienced  NLOS  gunners  ranging  in  height  from  5’8"  to  6’3". 


3.  Results: 

a.  Photographs  were  obtained  of  gunners  in  the  vehicle  with  the  sheet  metal  mock-up 
for  further  review. 


b.  The  following  observations  were  made  while  reviewing  the  sheet  metal  mock-up: 

-  Gunners  can  reach  all  the  gunner  console  control  switches  on  the  proposed 
Boeing  gunner  console 

-  Gunners  reported  that  the  on-the-dashboard  position  of  the  gunner  console  screen 
provides  an  acceptable  viewing  angle 

-  Gunners  and  other  members  of  the  NIjOS  community  commented  that  the  right 
armrest  and  joystick  might  be  broken  or  danced,  and  that  the  joystidc  in  particular 
might  get  "knocked  around"  and  need  recalibration.  One  gunner  was  observed  to  bend 
the  armrest  sideways  as  he  moved  about  to  adjust  his  seatbelt  The  gunners  and  others 
said  that  the  right  armrest  and  joystick  could  be  a  maintenance  problem  due  to  a 
potential  for  damage  and  the  trouble  of  tracking  down  armrest  wiring  related  problems. 
Gunners  and  others  recommended  that  we  ejq>lore  ways  of  integrating  the  joystick  with 
the  other  components  of  the  gunner  console. 

-  Gunners  wore  LBE  (with  only  one  of  the  two  canteens)  with  the  armrests  in  the 
down  position  and  reported  that  the  gunners  position  was  cramped  and  allowed  for  little 
movement  Gunners  stated  that  wearing  LBE  in  the  gunners  position  is  an  artificial 
evaluation,  as  soldiers  will  remove  their  LBE  and  store  it  in  the  cab  when  the  vehicle  is 
moving. 

-  The  on-the-dashboard  location  of  the  gunner’s  console  blocks  the  drivers  view  of 
the  right  side  vehicle  rear  view  mirror  and  also  partially  blocks  the  drivers  view  of  the 
ground  near  the  right  front  of  the  vehicle. 

-  The  on-the-dashboard  location  of  the  gunner’s  console  might  block  the  windshield 
defrost  vent 
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c.  The  following  observations  were  made  while  reviewing  the  gunner  console  using  the 
Styrafoam  mock>up  while  driving  over  dirt  roads  at  Training  Area  #30: 

-  The  gunners  did  not  hit  against  the  gunner  console  screen  during  low  speed 
movement  over  dirt  roads  with  some  bumps.  Most  gunners  and  others  did  bump  the 
front  of  their  hat  brims  directly  against  the  gunners  console  screen  under  a  hard  breaking 
maneuver.  The  breaking  maneuver  involved  moving  at  10  mph  on  a  level  dirt  road, 
announcing  that  breaking  was  to  occur,  and  then  hard  breaking  which  did  not  necessarily 
lock  up  the  vehicle  tires. 

-  The  two  gunners  about  5’8"  tall  reported  that  they  couldn’t  see  where  they  were 
going  because  the  gunner’s  console  was  in  the  way.  Comments  were  made  that  for 
daytime  movement,  the  gunner’s  console  digital  map  was  an  additional  aid,  and  that  the 
paper  map  was  still  the  primary  and  most  reliable  source  of  information  to  the  guimer. 
Guimers  said  that  the  gunner,  not  the  driver,  is  responsible  for  using  the  paper  map,  and 
that  they  can’t  use  the  map  if  they  can’t  see  where  they  are  going.  Gunners  said  that 
both  crew  members  needed  to  see  for  safe  night  movement  The  guimer  needs  an 
unrestricted  forward  view. 

-  Gunners  pointed  out  that  soldiers  may  not  be  willing  to  wear  a  shoulder  harness 
portion  of  a  modified  seat  belt  arrangement  which  Boeing  has  suggested  as  a  means  of 
restricting  guimer  forward  movement  into  the  gunner’s  console.  The  concern  here  is  that 
provision  of  a  modified  restraint  system  may  not  be  a  sufficient  measure  to  prevent 
accidents  involving  gunners  impacting  against  the  gunner  console  screen.  T^  is  similar 
to  the  observation  that  Gunners  will  likely  not  wear  LBE  during  vehicle  movement. 


d.  The  following  observations  were  made  while  reviewing  the  gunner’s  console 
using  the  three-piece  plywood  mock-up  while  stationary; 

-  The  gunners  stated  that  the  positioning  of  the  gunner  console  screen 

directly  in  front  of  the  gunner  on  the  dash  was  correct  for  conducting  engagements  but 
not  as  a  land  navigation  aid  during  vehicle  movement  Land  navigation  and  night 
movement  requires  that  the  gunner  be  able  to  see  out  the  windshield. 

-  The  gunners  did  not  like  the  idea  of  moving  the  gunner  console  screen  under  the 
dash  similar  to  the  NLOS  lOE  configuration. 

-  The  gunners  would  like  to  see  the  two  control  pannels  moved  down  and  out  of 
the  window  viewing  area. 

-  The  gunners  stated  that  moving  the  gunner’s  console  screen  to  a  lower  position 
directly  in  front  of  the  gunner  was  desirable  during  vehicle  movement.  Gunners 
suggested  that  this  might  be  accomplished  using  an  T"  shaped  bracket  that  would  allow 
the  screen  to  be  pulled  several  inches  toward  the  gunner  and  then  down  into  a  chest 
level  position  during  vehicle  movement. 
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-  Alternately,  the  screen  could  be  moved  laterally  toward  the  driver  during  vehicle 
movement  A  mounting  for  the  gunner’s  console  similar  to  that  used  on  the  lOE 
NLOS  fire  unit  might  be  adopted,  allowing  the  gunner’s  console  to  slide  one  and  a  half 
feet  to  the  left  fi'om  a  position  directly  in  front  of  the  gunner  to  a  position  out  of  his 
direct  field  of  view  through  the  windshield. 


4.  Siunmaiy 

-  The  gunners  stated  that  the  positioning  of  the  gunner  console  screen 

directly  in  front  of  the  gunner  on  the  dash  was  correct  for  conducting  engagements,  but 
not  as  a  land  navigation  aid  during  vehicle  movement  Land  navigation  and  night 
movement  requires  that  the  gunner  see  out  the  windshield. 

-  A  road  test  with  the  gunner’s  console  mock-up  suggests  that  gunners  wearing  lap 
seatbelts  can  be  thrown  into  the  console  screen.  The  addition  of  an  improved  restraint 
^tem  might  aleviate  this  potential  problem,  if  gunners  can  be  expected  to  wear  a 
shoulder  harness  for  extended  period  of  time. 

-  Gunners  did  not  like  the  idea  of  placing  the  gunner’s  console  screen  under  the  dash 
in  a  configuration  similar  to  that  of  the  current  lOE  systems. 

•  Gunners  suggested  that  we  look  into  mounting  the  gunner’s  console  screen  so  that  it 
can  be  moved  down  and  out  of  the  window  area  during  vehicle  movement,  or  sideways 
to  the  left  during  vehicle  movement  Proximity  of  controls  to  displays  is  a  valuable 
design  feature,  but  safety  concerns  and  a  need  for  gunner  forward  vision  may  represent  a 
higher  priority. 

-  Armrests  and  their  controls  raised  concerns  regarding  potential  for  damage  and  a 
resulting  maintenance  requirement 


S.  Additional  Arnty  Research  Institute  concerns; 

-  Consider  the  implications  of  having  a  TV  type  screen  mounted  above  dashboard 
level  in  a  HMMWV  in  terms  of  night  operations  light  discipline  requirements. 

-  Consider  the  implications  of  a  dashborad  mounted  TV  type  screen  light  source  on 
the  driver  and  gunner’s  night  vision  and  driving  ability. 

-  Boeing  gunner’s  console  design  places  the  control  box  with  the  time  critical  data 
entry  and  engagement  switches  on  the  left  side  of  the  console,  but  most  gunners  are  right 
handed.  Would  human  performance  be  enhanced  by  moving  the  data  entiy  box  over  to 
the  right  side  of  the  console? 
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6.  The  POC  for  this  action  is  Mr.  Bill  Sanders,  AV978-4491/5297. 


End 

as 


WnUAM  SANDERS 
Research  Psychologist 
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APPENDIX  B 


NLOS  EUE  CREWMEMBER  EGRESS  EVALUATION 
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DEPARTMENT  OF  THE  ARMY 
U.S.  Arn^  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Fort  Bliss  Field  Unit,  P.O.  Box  6057 
Fort  Bliss,  Texas  79906^7 


PERI-SB-C  30  August  1990 

MEMORANDUM  FOR  DIR,  TRADOC  SYSTEM  MANAGER,  ATSA-TSM-F,  ATTN: 
MAJFEDAKO 

SUBJECT:  NLOS  EUE  CREWMEMBER  EGRESS  EVALUATION 

1.  Purpose 

Conduct  an  evaluation  to  assess  how  the  addition  of  equipment  to  the  NLOS  crew 
cab  might  impact  the  ability  of  both  aewmembers  to  egress  the  vehicle,  and  identify  key 
design  features  that  facilitate  or  restrict  egress  performance.  The  proposed  addition  of 
several  items  of  equipment  to  the  crew  cab  of  &e  NLOS  fire  unit  raised  Hmnan  Factors 
Engineering  and  System  Safety  MANPRINT  concerns  regarding  the  crewmembers  ability 
to  quickly  exit  the  vehicle  through  a  single  door  in  the  event  of  overturning,  or  a  serious 
accident  No  crewmember  egress  performance  requirement  has  been  established  for 
NLOS,  however  Human  Factors  Engineering  MIL  STD  1472D  provides  some  design 
standards. 

2.  Method 

The  evaluation  was  conducted  by  Arnty  Research  Institute  Fort  Bliss  Field  Unit 
within  the  framework  of  MANPRINT  support  to  the  NLOS  Extended  User  Employment 
effort  Four  NLOS  gunners  from  A  battery,  2nd  Battalion,  6th  ADA  Brigade,  were 
evaluated  on  their  ability  to  egress  fi'om  an  NLOS  fire  unit  Performance  information 
was  recorded  using  a  stop  watch  and  videotape  recordings.  To  sirmilate  a  vehicle  turned 
on  its  side  both  crewmen,  driver  and  guimer,  were  asked  to  exit  through  a  single  door. 
Crewmen  were  not  allowed  to  assist  each  other  in  exiting  the  vehicle.  Trials  began  with 
both  crewmen  wearing  their  seat  belts.  Crewmen  exited  the  vehicle  through  the  left 
door,  and  the  right  door,  in  both  MOPPO  and  MOPP4  daylight  conditions. 

a.  Trials  Schedule:  The  four  gunners  were  divided  into  two  teams.  Each  team 
completed  trials  in  MOPPO  and  MOPP4,  first  exiting  through  the  right  door,  and  then 
the  left  Eadi  gunner  attenq)ted  four  trials  in  eadi  of  the  four  conditions.  Right  MOPP4, 
Right  MOPPO,  Left  MOPP4,  Left  MOPPO,  for  a  total  of  sixteen  trials  per  gunner,  and 
sixty-four  trials  total.  Team  members  rotated  between  driver  and  passenger  positions 
after  each  trial  Teams  rotated  after  completion  of  the  sixteen  trials  in  each  condition. 
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b.  Equipment  Mock-Ups:  Three  mock-ups  were  built  to  represent  key  components  of 
the  proposed  Boeing  NLOS  system  which  might  impact  crewmember  egress  performance, 
a  Gunner’s  Console,  Air  Plemun  Box,  and  Enhanced  Handheld  Terminal  Unit 
Components  were  installed  in  Fire  Unit  #1,  with  the  original  Gunner’s  Console  removed 
and  ^e  overhead  ventilation  box  and  hoses  removed  to  more  closely  resemble  the 
objective  NLOS  configuration.  The  fire  extinguisher  was  relocated  4"  forward  of  original 
position  between  seats  to  allow  for  the  placement  of  the  Air  Plenum  Box. 

1.  Gunners  Console:  The  Full  Scale  Development  (FSD)  gunner’s  console  mock-up 
was  composed  of  three  rectangular  sheet  metal  boxes  measuring  12  X  5  X  8.75  inches,  8 
X  11.25  X  10  inches,  and  12  X  5  X  5  inches,  generally  meeting  the  dimensions  presented 
in  the  updated  diagram  "Gunner’s  Console  Design  Concept  (Dimensional)"  provided  as 
enclosure  1. 

2.  Air  Plenum  Box:  An  Air  Plenum  box  has  been  proposed  for  the  FSD  NLOS 
crew  compartment  to  provide  cooling  air  to  the  NLOS  computer  equipment  The  box 
would  be  located  between  the  crew  compartment  seats,  fitting  flush  against  the  rear 
window  of  the  compartment  The  Air  Plenum  box  mock-up  was  a  rectangular  plywood 
box  measiiring  24  X  25  X  10  inches,  covered  with  fabric  to  reduce  any  surface  snag 
potential 

3.  EHTU:  An  Enhanced  Hand-held  Terminal  Unit  (EHTU)  has  been  proposed  for 
the  NLOS  system,  and  diagrams  have  shown  this  equipment  mounted  above  other 
SINCGARS  and  Enhanced  Position  Location  Reporting  System  (EPLRS)  equipment 
For  this  evaluation  a  3  X  12  X  11.5  inch  rectangular  EHITJ  mock-up  was  constructed 
from  a  hard  cover  three  ring  binder  capable  of  supporting  the  full  weight  of  a  soldier. 
The  mock-up  represented  an  EHTU  with  the  screen  lid  in  the  closed  position.  The 
EHTU  mock-up  was  secured  in  place  on  top  of  the  NLOS  radio  rack  with  duct  tape. 

c.  Gunner’s  personal  gear:  Each  gunner  wore  BDU’s,  kevlar  helmet,  web  gear,  two 
canteens  (empty)  on  a  belt  For  MOPP4  trials  gunners  wore  MOPP4  overgarment, 
mask/hood,  ^oves,  overboots,  web  gear,  canteens  on  a  belt  Gunners  did  not  wear  a 
flack  jacket  MOPP4  trials  for  the  two  crewmen  evaluated  in  the  afternoon  were 
modified  so  that  the  inner  liner  of  the  MOPP4  suit  was  not  worn.  The  liner  was  not 
worn  because  it  became  saturated  with  moisture  during  early  trials.  Soldiers  were 
allowed  to  remove  MOPP4  gear  between  trials  to  cool  off. 

d.  Soldier  demographics:  Crewmen  ranged  from  5’8"  to  6’2.5"  in  height  and  were  all 
of  light  build  ranging  from  145  to  185  pounds  in  weight 


3.  Results 

a.  Available  Passage  Space:  With  the  Gunner’s  Console,  Air  Plenum,  and  EHTU 
mock-ups  in  place,  the  av^able  passage  space  near  the  roof  in  the  NLOS  crew  cab 
between  the  driver  and  gunner  positions  was  approximately  12  -  13  inches  high  and  32 
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inches  wide.  However,  roof  support  bars  reduced  the  available  passageway  height  to 
only  10-11  inches  directly  beneath  these  one  inch  wide  supports.  Hiere  was  also  an 
adjoining  6  inch  wide  and  30  inch  high  passageway  open  between  the  radio  rack  and  the 
air  plenum  extending  from  the  floor  above  the  fire  extinguisher  to  the  roof  (see 
enclosure  2).  MIL  STD  1472D  section  S.7.83  (Whole-body  access)  states  that 
dimensions  for  rectangular  access  openings  for  body  passage  of  soldiers  in  light  clothing 
shall  not  be  less  than  26  inches  high  and  30  inches  wide,  and  for  bulky  clothing  not  less 
than  29  inches  high  and  34  inches  wide,  and  that  diameters  of  oval  hatches  in  armored 
vehicles  shall  not  be  less  than  17  inches  by  28  inches  (see  enclosure  3). 

b.  Time  Data  Analysis  Siunmaiy:  Times  reqiiired  for  gunners  to  egress  from  the 
stationary  vehicle,  from  a  "seat  belt  on"  position  to  the  time  the  gunner  had  both  feet  on 
the  groimd  and  body  fully  outside  the  vehicle  are  summarized  below: 

-  MOPPO  times  (in  seconds) 

-  Left  side  egress 

Mean:  14.41 

Minimum:  7.67 
Maximum:  2232 

-  Right  side  egress 

Mean:  12.% 

Minimum:  5.98 

Maximum:  24.81  (2  hang-ups  omitted) 


-  MOPP4  times  (in  seconds) 

-  Left  side  egress 

Mean:  1222 

Minimum:  9.10 
Maximum:  1737 

-  Right  side  egress 

Mean:  15.61 

Minimum;  8.04 

Maximum:  27.64  (3  hang-ups  omitted) 
Mean  performance  time  across  59  trials:  13.74  seconds 


c.  Observations 


1.  Use  of  lOE  NLOS  overhead  roof  support  bars  as  hand-holds:  The  NLOS  lOE 
^tems  have  a  canvas  roof  with  a  one-inch  foam  rubber  covered  roof  support  bar 
running  centrally  from  the  roof  above  the  drivers  side  door  to  the  roof  above  the 
passenger  side  door.  Two  one-inch  roof  support  bars  run  in  parallel,  six  inches  apart, 
centrally  from  the  front  edge  to  the  back  of  the  roof,  as  shown  previously  in  Enclosure  2. 
The  canvas  roof  above  these  bars  is  flexible  so  that  soldiers  can  close  their  hands  around 
the  bar  to  grasp  it  and  pull  themselves  out  of  the  vehicle.  It  was  found  that  these  roof 
support  bars  were  used  extensively  as  an  assist  to  gunner  egress.  The  first  attempt  by  a 
gunner  to  crawl  'helly  down"  over  the  radio  equipment  rack  resulted  in  a  great  deal  of 
struggling  and  effort  and  an  egress  time  of  about  one  minute.  After  seeing  this  trial 
another  guimer  adopted  the  technique  of  inverting  himself  "belly  up"  and  using  the 
overhead  support  bars  as  hand  holds  to  pull  himself  over  the  radio  rack  imtil  he  could 
reach  out  and  grab  the  edge  of  the  roof  above  the  open  doorway.  This  successful 
technique  of  using  the  overhead  support  bars  as  handholds  was  adopted  by  all  four 
gunners  for  subsequent  trials,  unless  they  were  told  to  try  the  "belly  down"  technique  by 
the  evaluators. 

2.  Radio  rack  to  Air  Plenum  passage  way  use:  A  six-inch  wide  and  thirty-inch  tall 
passage  way  existed  between  the  Air  Plenum  and  the  radio  rack  which  was  very  useful  to 
guimers  as  they  exited  the  vehicle.  The  passageway  is  partially  obstructed  by  the  wiring 
cables  protruding  from  the  face  of  the  SINCX5ARS  radios  toward  the  Air  Plenum-  This 
passageway  between  the  two  large  equipment  structures  was  used  by  gunners  during 
egress  to  drag  one  leg  through  beneath  their  boefy,  both  to  push  against  the  floor  and 
support  their  body  weight  as  they  moved  over  the  radio  equipment,  and  also  to  support 
their  body  once  they  had  cleared  the  radio  rack  and  needed  to  swing  out  of  the  vehicle. 

3.  lOE  system  soft  canvas  roof:  The  soft  "canvas"  type  roof  played  a  role  in  the 
egress  tasks.  It  was  clearly  visible  that  the  canvas  roof  was  being  pushed  up  as  gunners 
worked  to  crawl  through  the  narrow  passage  way  space  between  crew  compartment  Air 
Plenum  and  radio  rack  equipment  and  the  roof. 

4.  Guimers  adopted  the  practice  of  removing  their  helmets  and  throwing  them  over 
the  radio  equipment  rack  prior  to  attempting  egress. 

5.  Performance  times  for  five  trials  were  not  included  in  summaiy  statistics  because 
of  complete  egress  failure,  or  where  soldier  personal  gear  became  entangled  on  NLOS 
equipment  Observations  for  these  five  trials  are  detaUed  below: 

6.  Trial  la:  On  the  first  trial  the  gunner  attempted  a  "belly  down"  crawl  over  the 
radio  rack  equipment  The  gunner  searched  for  handholds  to  use  to  support  his  weight 
and  to  use  in  pulling  himself  over  the  equipment  The  soldier  tried  to  egress  with  his 
helmet  on.  The  gunner  succeeded  in  exiti^  after  56.94  seconds. 
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7.  Trial  7a:  The  soldier  became  stuck  and  was  not  able  to  egress.  The  gunner  got 
caught  in  the  space  between  the  radio  rack  and  the  Air  Plenum.  He  squeezed  his  body 
along  the  front  of  the  air  plenum  but  the  equipment  at  his  waist  pushed  directly  up 
against  the  drivers  side  of  the  air  plenum  and  would  not  allow  him  to  pull  through. 

8.  Trial  8b:  The  soldier  became  "hung-up”  as  the  ammunition  pouch  on  the  right 
side  of  his  body  became  entangled  on  the  windshield  wiper  box  knob.  The  gunner  was 
egressing  in  a  belly-up  position  and  had  his  right  hand  firmly  grasping  the  passenger  side 
roof  edge  when  his  equipment  became  entan^ed.  The  gunner  was  unable  to  pull  his 
equipment  off  the  wiper  box  switch  with  his  free  left  hand.  The  gunner  moved  back  into 
the  vehicle,  freed  the  gear  with  his  right  hand,  and  exited  the  vehicle  without  assistance 
after  a  total  of  35.13  seconds. 

9.  Trial  9a:  The  soldier  became  ”hung-up”  when  his  web  gear  became  entangled 
with  equipment.  The  gunner  successfully  egressed  from  the  vehicle  without  assistance 
after  46.43  seconds.  No  video  of  this  trial 


10.  Trial  19a:  The  guimer  tried  to  egress  in  a  belly-down  position  with  his  helmet 
on.  He  retreated  backwards  to  remove  his  helmet,  and  again  tried  a  belly-down  egress 
while  trying  to  hold  his  loose  personal  gear  dose  to  his  body.  The  personal  gear  got 
caught  on  equipment,  and  the  soldier  retreated  backwards  and  attempted  a  belly-up 
egress.  He  experienced  difficulty  again,  returned  to  the  starting  position  in  the  drivers 
seat,  and  announced  Tm  dead”  after  3824  seconds  time  had  elapsed.  The  gunner  felt 
he  could  do  much  better  and  wanted  to  end  this  trial  when  it  was  clear  he  could  not 
egress  quickly.  In  his  next  two  trials  he  egressed  successfully  in  ten  seconds  or  less. 

11.  Guimer  performance  motivation:  Gunners  displayed  a  high  level  of  motivation 
to  achieve  low  egress  performance  times  during  the  trials.  An  informal  competition 
occurred  between  the  guimers  to  achieve  the  lowest  time. 

12.  Force  exerted  on  equipment  mock-ups:  The  gunners  exerted  a  great  deal  of 
force  on  some  equipment  mock-ups  during  their  efforts  to  egress  from  the  NLOS  system. 
The  GC  mock-up  was  knocked  sideways  off  the  dashboard  of  the  vehicle  and  had  to  be 
re-secured  with  additional  duck  tape.  The  plastic  screen  on  the  GC  was  broken  off 
during  an  early  trial.  The  guimers  stepped  on  the  fire  extinguisher  during  egress  and 
moved  it  aroimd  within  its  fixture.  The  EHTU  mock-up  had  to  be  re-secured  with 
additional  duck  tape  during  the  course  of  the  trials  as  a  great  deal  of  force  was  exerted 
as  the  gunners  pulled  themselves  over  this  equipment  and  out  of  the  vehicle. 


4.  Summaiy 

a.  Egress  was  evaluated  because  the  proposed  addition  of  several  items  of  equipment 
to  the  crew  cab  of  the  NLOS  fire  unit  raised  the  concern  that  the  crewmembers  ability  to 
quickly  exit  the  vehicle  might  be  restricted. 
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b.  No  specific  crewmember  egress  performance  requirement  had  been  specified  for 
NLOS,  however  MIL  STD  1472D  (Whole-body  access)  provides  related  desi^  standards. 
The  NLOS  lOE  system  with  equipment  mock-ups  installed  offered  an  approximately  11 
inch  by  32  inch  radio  rack  to  roof  passage  way  for  egress.  This  passage  way  was 
considerably  smaller  than  the  MIL  STD  1472D  recommendation  of  29  inches  by  34 
inches  for  crewmembers  wearing  buU^  clothing. 

c.  The  gunners  were  able  to  egress  over  the  radio  equipment  rack  from  both  sides  of 
the  vehicle,  in  both  MOPPO  and  MOPP4.  No  differences  appear  between  MOPPO  and 
MOPP4  performance  times,  or  between  left  and  right  side  egress  times.  Mean  egress 
timgi  across  59  successful  trials  was  13.74  seconds.  Egress  failures  or  extreme  times 
occurred  on  five  trials  which  were  not  included  in  overall  time  computations. 

d.  Gunners  most  frequently  used  the  roof  support  rails  to  pull  themselves  upside 
down  over  the  radio  rack.  However,  guimers  al^  were  successful  using  an  "arms  first" 
belly  crawling  maneuver  to  get  over  the  radio  rack  to  egress  from  the  vehicle,  so  that  the 
roof  rails  were  not  absolutely  essential  for  successful  egress. 

e.  Gunners  made  use  of  the  available  passageway  space  between  the  SINCGARS 
radio  rack  and  the  Air  Plenum  Box.  Aoy  Full  Scale  Development  design  should  consider 
the  utility  of  preserving  this  six  inch  passageway  as  a  minimum  to  facilitate  emergency 
egress. 

1  Gunner  egress  time  trends  suggest  that  knowledge  of  an  effective  approach  to 
egress  and  practice  can  greatly  reduce  egress  times. 


5.  Additional  Concerns 

a.  The  close  (6  inch)  proximity  of  the  Air  Plenum  Box  facing  the  SINCGARS  radios 
made  reading  the  SINGCARS  displays  difBcult,  and  could  prevent  the  removal  of  the 
SINGCARS  from  the  current  lOE  r^o  rack.  SINCGARS  display  visibility  and  removal 
requirements  need  to  be  incorporated  into  the  Full  Scale  Development  system  design. 

b.  Several  weeks  after  the  evaluation  an  NLOS  gunner  stated  that  crewmembers 
would  also  normally  be  required  to  wear  flack  jackets  in  an  operational  environment 
Flack  jackets  were  not  worn  during  the  egress  evaluation.  The  addition  of  a  flack  jacket 
to  personal  gear  worn  by  the  gunners  could  make  egress  a  much  more  difficult  task  and 
si^iificantly  inaease  performance  times.  Egress  from  the  vehicle  might  not  have  been 
possible  if  gunners  were  wearing  flack  jackets.  Removal  of  flack  jackets  would  require 
some  time  in  the  cramped  crew  compartment 

c.  A  hardtop  version  of  the  NLOS  would  not  provide  the  overhead  bars  used 
extensively  by  soldiers  to  pull  themselves  out  of  the  vehicle,  and  would  not  provide  the 
flexible  roof  characteristic  that  allowed  the  roof  to  be  raised  several  inches  so  that 
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gunners  had  additional  room  to  squeeze  over  the  equipment  Egress  performance  could 
be  significantly  worse  if  the  soft  top  were  simply  repla^  with  a  hard  top  with  other 
equipment  remaining  the  same. 

6.  Preliminary  results  of  this  evaluation  were  presented  at  the  NLOS  Human  Factors 
Engineering  Working  Group  meeting  held  at  (he  Boeing  Huntsville  facility  on  4  June 
1990. 


7.  The  PCX;  for  this  action  is  Mr.  William  Sanders  or  Mr.  Don  Carter,  AV978- 
4491/5297. 


End  WILLIAM  SANDERS 

as  Research  Psychologist 
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GUNNER’S  SEAT 


NLOS  CREW  COMPARTMENT  WITH  MOCK-UPS  (GUNNER’S  SIDE  VIEW) 


TOP  AND  BOTTOM 
ACCESS 


NOTE ;  DIMENSIONS  SHOWN  BASED  ON  MALE  DATA. 


FIGURE  37.  WHOLE  BODY  ACCESS  OPENING 
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APPENDIX  C 

NLOS  EMBEDDED  TRAINER  EVALUATION 


DEPARTMENT  OF  THE  ARMY 


U.S.  Arn^  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Fort  Bliss  Field  Unit,  P.O.  Box  6057 
Fort  Bliss,  Texas  79906-0057 


PERI-SBC  (70-n)  22  February  1991 

MEMORANDUM  THROUGH  DIR,  ARI  FAAD  MANPRINT  TASK  FORCE  (ATTN: 
MAJ  LEVITT) 

TO:  TRADOC  SYSTEM  MANAGER  -  FAAD,  ATSA-TSM-F,  (ATTN: 

MAJ  FEDAKO) 

SUBJECT:  NLOS  EMBEDDED  TRAINER  MANPRINT  EVALUATION 


1.  PURPOSE:  Evaluate  NLOS  Initial  Operational  Evaluation  (lOE)  system  Embedded 
Trainer  (ET)  system  within  the  NLOS  Extended  User  Employment  (EUE)  program  to 
identify  soldier  tasks,  task  performance  requirements,  and  performance  trends.  Compare 
the  NLOS  ET  chara^ristics  to  ARI  developed  Embedded  Training  Test  and  Evaluation 
guidelines.  Develop  a  preliminary  NLOS  lOE  ET  engagement  task  performance  data 
base  in  order  to  fu^^er  e:q)lore  soldier  task  performance  trends  and  engagement 
outcomes  under  various  conditions. 


2.  RATIONALE:  The  NLOS  lOE  fire  unit  ET  system  task  and  human  performance 
evaluation  is  part  of  the  EUE  MANPRINT  program  effort  to  identify  soldier  tasks,  and 
task  performance  requirements  early  in  NLOS  ^tem  development  The  NLOS  System 
MAI^RINT  Management  Plan  (SMMP)  specifies  objectives  and  concerns  related  to 
task  skill  and  iqrtitude  requirements,  as  weU  as  specific  issues  regarding  the  NLOS 
Embedded  Training  system.  Current  MANPRIOT  NLOS  lOE  Embedded  Training 
efforts  are  directed  at  identifying  concerns,  gathering  data,  and  malring  evaluations  to 
answer  the  issues  outlined  in  the  SMMP,  as  well  as  supporting  other  information 
requirements  as  they  arise. 


3.  NLOS  ET  BACKGROUND:  ET  has  been  described  as  that  training  which  results 
from  features  incorporated  into  the  end  item  equipment  to  provide  training  and  practice 
using  that  end  item  equipment  With  the  NLOS  lOE  system  the  features  are  completely 
embedded  within  the  system  configuration  by  a  software  iq>plication.  ET  features  must 
include  stimuli  necessary  to  support  training,  and  should  provide  (1)  a  performance 
assessment  capability,  (2)  appropriate  feedback,  and  (3)  record  keeping.  Amty  policy 
dictates  that  ET  be  the  first  alternative  reviewed  for  providing  training  in  these 
categories:  individual  operator  or  maintainor,  crew,  functional  area,  and  force  level. 
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For  the  NLOS  lOE  ET  system  the  gunner  first  selects  the  "Simulation  Launch"  option 
from  the  Gunner’s  Console  main  menu,  and  then  has  the  opportunity  to  select  simulated 
NLOS  target  engagement  scenarios  from  either  a  Tixed"  or  "Variable"  scenario  menu. 
After  following  the  launch  control  procedure  and  enabling  the  fire  switch,  the  simulated 
launch  begins.  The  Digital  Map  Generator  (DMG)  provides  simulated  terrain  through 
the  implementarion  of  a  database,  and  updates  from  the  missile  processor  to  the  DMG 
via  the  Console  Processor  provides  the  illusion  of  "flying  over"  the  terrain.  The  gunner 
can  move  the  joystick,  changing  the  seeker  view  or  manually  guiding  the  missile,  just  as 
in  a  real  flight  situation,  and  the  DMG  terrain  simulation  will  respond  accordingly. 

When  using  the  Tixed"  scenarios,  the  launch  point  is  automatically  set  (which  may  be 
different  from  the  gunner’s  actual  vehicle  position)  and  a  pre-determined  target  area  is 
displayed.  This  is  the  target  the  gunner  must  hook  if  he  wishes  to  engage  the  target 
described  in  the  scenario  menu.  If  the  target  area  represents  a  moving  air  target,  then 
this  target  area  is  an  intercept  point  designed  to  take  the  missile  into  the  path  of  the 
target  •  it  is  not  the  initial  position  of  the  target 

The  "Variable"  scenarios  are  more  like  real  tactical  situations.  The  gunner  refers  to 
detailed  Scenario  Maps  before  selecting  a  specific  scenario.  The  Variable  scenarios, 
unlike  the  Fixed  scenarios,  will  allow  the  gunner  to  use  any  pre-planned  target  areas  or 
routes.  ^  moving  the  launch  site,  the  guimer  can  practice  engaging  a  target  from 
different  approaches.  Also,  the  system  will  randomly  select  half  of  the  targets  available 
for  a  particular  scenario  for  actual  presentation  during  the  engagement,  which  increases 
variability.  For  both  Fixed  and  Variable  scenarios,  the  targets  do  not  start  moving  until 
the  gunner  flips  the  TIRE  ENABLE"  switch. 


4.  METHOD: 

a.  Collect  soldier  Embedded  Trainer  task  performance  data  on  NLOS  lOE  ET 
using  video  t^,  analyst  observations,  and  after  action  debriefs. 

-  Conduct  a  pilot  data  collection  effort  to  identify  key  task  performance 
variables  and  develop  data  collection  tools. 

-  Collect  data  on  each  of  the  26  Fixed  Scenarios. 

-  Collect  data  on  each  of  the  16  Variable  Scenarios. 


b.  Review  video  frq>es  to  identify  start  and  stop  times  for  tasks,  range  to  target  area, 
and  error  information  for  a  sample  of  scenario  trials. 


c.  Develop  a  preliminary  MicroSAINT  soldier  performance  computer  model 
incorporating  NLOS  lOE  CT  tasks,  and  task  performance  time  data  in  order  to  further 
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explore  soldier  task  networks.  For  ocample,  a  task  such  as  target  identification  could  be 
given  various  timfe  and  error  performance  values  and  entered  into  the  model  to  explore 
what  impact  it  might  have  on  the  time  and  range  to  target  at  which  other  tasks  must  be 
performed. 


d.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  and  the 
Project  Manager  for  Training  Devices  (PM  TRADE)  has  produced  a  series  of  ten 
related  documents  sharing  the  general  title  of  Implementing  Embedded  Training  fET^. 
This  series  of  documents  includes  guidelines  and  procedures  that  support  the  effective 
consideration,  definition,  development,  and  integration  of  embedded  training  (ET) 
C£q>abilities  for  existing  and  developmental  ^tems.  Volume  7  of  this  series. 

Embedded  Training  Test  and  Evaluation,  identifies  eleven  Required  Technical 
Characteristics  for  Embedded  Training  systems.  The  NLOS  EUE  MANPRINT  support 
effort  has  addressed  these  eleven  characteristics  in  an  effort  to  provide  a  systematic  and 
comprehensive  assessment  of  the  system. 


5.  RESULTS: 

a.  Soldier  NLOS  ET  task  performance  data  collection: 

•  The  pilot  data  collection  effort  was  conducted  12  June  1990  at  Ft  Bliss.  Two 
experience  NLOS  guimeis  described  their  actions  as  they  conducted  several  ET 
engagements.  A  video  camera  positioned  over  the  right  shoulder  of  the  gunner  recorded 
both  audio  and  video  data  for  the  engagements.  In  the  video,  the  gunners  describe  what 
they  are  doing,  and  why,  as  they  walk  the  viewer  through  the  ET  ^tem  to  include  all 
button  pushes,  scenario  selection  from  the  menu,  and  engagement  events  performance. 

As  an  example  of  the  supporting  information  available  for  each  scenario,  i^pendix  1 
provides  the  target  movement  data  and  engagement  map  for  scenario  #16  (attack 
against  a  fom  helicopter  squadron). 

-  Fixed  scenario  task  performance  was  gathered  at  Ft  Bliss  27-28  June,  1990.  Each  of 
the  26  Fixed  scenarios  were  video  Uq)ed  on  VHS  format  as  four  NLOS  gunners  carried 
out  engagements.  Each  gunner  conducted  either  the  scenarios  1-13,  or  14-26,  and  an 
additional  set  of  four  scenarios  (#8,  #11,  #16,  #18),  yielding  a  total  68  ET  scenario 
trials  in  the  video  tape  data  base.  A  sample  of  12  of  the  68  trials  was  reduced  to  obtain 
task  time  and  range  to  target  area  data.  Information  from  the  remaining  trials  will  be 
reduced  as  necessary  to  meet  any  additional  requests  for  scenario  specific  information. 

-  Variable  scenario  data  collection  was  conducted  20-22  September,  1990.  Problems 
were  encountered  as  it  was  not  clear  how  to  effectively  exercise  the  16  Variable 
scenarios.  A  single  run  through  of  each  scenario  was  recorded.  Mai^r  of  the  key 
features  and  c£q)abilities  of  the  Variable  scenario  ET  software  were  not  exercised  or 
explored.  Proper  use  of  the  Variable  scenarios  required  that  the  gunner  have  the  full 
size  paper  maps  displaying  target  positions  which  were  provided  for  each  of  the  16 
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scenarios.  Gunners  would  use  the  maps  to  dedde  on  where  to  position  the  fire  unit,  and 
the  target  coordinates  to  fire  on.  Gunners  were  not  confident  about  their  ability  to  carry 
out  the  Variable  scenarios,  and  were  discouraged  when  they  were  unable  to  find  targets 
during  preliminaiy  engagement  trials. 

b.  VideoUqpes  were  reviewed  to  identify  soldier  performance  tasks  associated  with  the 
lOE  Embedded  Training  system.  A  preliminary  task  list  reflecting  task  performance 
fi-om  the  pilot  study  videotr^re  was  created  (see  Appendix  2).  A  streamlined  task  list 
focusing  on  the  engagement  was  used  to  record  task  time,  and  range  to  target  area  data 
from  the  videoUq)ed  scenario  performance  data  (see  Appendix  3).  Task  performance 
times  and  ranges  for  scenarios  #6,  #11,  #16,  and  #18  were  compiled  from  the 
videotaped  data  for  preliminary  evaluation  of  gurmer  task  performance  trends  (Tables 
1*4).  data  reduced  from  these  four  scenarios  was  used  to  illustrate  the  time-stressed 
human  task  performance  demands  of  the  NLOS  system  (Figure  1),  which  would  also 
include  a  requirement  for  target  identification.  The  breakdown  of  performance  data 
provides  a  fint  empirical  look  at  how  gurmer  performance  differs  both  within  and 
between  engagement  scenarios.  This  is  a  first  step  toward  identifying  measurable 
performance  standards  or  criteria  for  each  scenario.  The  identification  of  key  tasks  and 
performance  standards  is  a  prerequisite  for  the  development  of  automated  performance 
scoring  schemes,  and  feedback  displays  for  ET  or  other  training  systems.  Sample  ET 
performance  feedback  displays  will  be  presented  later  in  this  paper. 


c.  A  preliminary  MicroSAINT  soldier  performance  computer  model  was  developed 
by  ARI  Ft  Hood  and  ARI  Ft  Bliss  incorporating  NLOS  lOE  ET  tasks,  and  task 
performance  times.  MicroSAINT  is  a  ^tem  that  lets  users  develop,  execute,  and 
analyze  the  results  of  network  simulation  models.  In  the  case  of  NLOS  ET, 
MicroSAINT  is  used  to  explore  soldier  performance  trends  and  engagement  outcomes 
under  various  conditions,  such  as  changes  in  task  performance  time  or  reliabilify. 
Estimates  of  gunner  task  performance  from  a  quick  review  of  the  trials  on  Fixed 
Scenarios  #6,  #11,  #16,  and  #18  were  entered  into  the  preliminary  model.  The  model 
randomizes  task  performance  across  trials  and  yields  new  combinations  of  existing  trial 
data.  An  example  of  the  NLOS  ET  engagement  task  timeline  output  provided  by  the 
MicroSAINT  model  is  presented  as  Figure  2. 


d.  ARI  Research  Product  89*02  identifies  eleven  Required  Technical  Characteristics 
for  embedded  training  ^tems.  The  NLOS  lOE  ET  ^tem  was  evaluated  in  terms  of 
these  characteristics  in  an  effort  to  provide  a  systematic  and  conq)rehensive  assessment 
of  the  system.  The  review  of  the  ET  system  against  the  eleven  characteristics  is 
summarized  in  Figure  3.  The  detailed  review  of  the  lOE  NLOS  system’s  Embedded 
Trainer  with  regard  to  each  of  the  eleven  Required  Technical  Characteristics  follows: 
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'IABI£  1 


ENGMEMElfr  TftSK  PEI^POFMANCE  CXMIIATEVE  TIME,  AND  RANGE  TO  TARGET  AREA 
SCENARIO  #6  (IVTO  Heliocpters  Crossing) 

time ^SEOCTTOS^ /RANGE  TO  TARGET  AREA  (METERS) 


TASK 

TRIAL  1 

TOIAL  2 

TRIAL  3 

1.  "Enable  Fire" 

0/5150 

0/5150 

0/5150 

2.  "Auto  Search"  Start 

8/4500 

8/4500 

8/4500 

3.  Zoon  Seeker  Manually  (1st) 

36/1480 

10/4200 

18/3458 

4.  Slew  Seeker  Manually  (1st) 

11/4143 

12/3900 

22/3086 

5.  Target  In  Field  Of  View 

7/4560 

12/3900 

15/3880 

6.  Start  Seeker  Slew  To 

11/4143 

13/3800 

16/3500 

Target  (Target  Detected) 

7.  End  Seeker  Slew 

30/2229 

18/3310 

34/1640 

8.  First  "Lock-On"  Try 

30/2121 

40/1200 

36/1481 

9 .  First  "Break-lock" 

30/2067 

None 

48/* 

10.  Second  "lock-On"  Try 

33/1906 

— 

1:49/* 

11.  Second  "Break-lock" 

None 

— 

None 

12.  Target  Identified 

Don't  Kxw 

Don't  Rtcw 

Don't  Know 

13.  Missile  Inpacts  Target 

48/Hit 

48/Hit 

2: 02/Hit 

14.  Trial  1  Ocnroents:  None. 

15.  Trial  2  Ccniinents:  None. 

16.  Tried  3  Oomments:  Missile  flew  by  target  at  48 
turn  and  re-engaged  target  successfully  at  2:02. 

seconds,  gunner 

inade  a  U- 

♦Missile  reached  origincil'  target  eonea  euid  distcince  to  target  indicator  stopped 
at  499. 
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table  2 


ENGAGEMENT  TASK  FERPC»MANCE  CUMUIATIVE  TIME,  AND  RANGE  TO  TARGET  AREA 
SCENARIO  #11  (Short  Fli^t  To  Tank  On  Road) 


ENGAGEMENT  TIME rSEOONDS^ /RANGE  TO  TARGET  AREA  fMETE3^) 


TASK 

TRIAL  1 

TRIAL  2 

TRIAL  3 

1.  "Enable  Fire" 

0/3842 

0/3842 

0/3842 

2.  "Auto  Search"  Start 

Immediate 

Immediate 

Immediate 

3.  Zoom  Seeker  Manually  (1st) 

16/2320 

7/3300 

18/2300 

4.  Slew  Seeker  Manually  (1st) 

10/3000 

7/3300 

11/3000 

5.  Target  In  Field  Of  View 

11/2845 

5/3570 

Never 

6.  Start  Seeker  Slew  To 

Target  (Target  Detected) 

14/2583 

8/3240 

— 

7.  End  Seeker  Slew 

30/879 

23/1619 

— 

8.  First  "Lock-On"  Try 

31/664 

25/1244 

— 

9 .  First  "Break-Lock" 

None 

29/977 

— 

10.  Second  "Lock-On"  Try 

— 

None 

— 

11.  Second  "Break-Lock" 

— 

— 

— 

12.  Target  Identified 

Don't  Kxw 

Don't  Yjtcm 

Don't  Knew 

13.  Missile  Reaches  Target  Area 

39/Hit 

33/Miss 

34/Miss 

14.  Trial  1  Comments:  Good  detection,  lock-on,  and  fly  to  irpact. 


15.  Trial  2  Comments:  Gunner  pinned  seeker  looking  at  tcirget,  but  not  guiding 
the  missile  to  the  target,  over  flew  target,  U-tum,  crashed  in  turn  at  46 
seconds. 


16.  Trial  3  Comments:  Gunner  immediately  pinned  seeker  left,  never  got  a 
target  in  this  left  direction  field  of  view,  U-tum,  crai^ed  at  2:33. 


C-7 


TABLE  3 


ENGAGEMENT  TASK  PE3?Ft»MANCE  CUMULATIVE  TIME,  AND  RANGE  TO  TARGET  AREA  - 
SCENARIO  #16  (Four  Helicopter  Squadron) 


ENGAGEMENT  TIME  ^SECONDS  > /RANGE  TO  TARGET  AREA  fMETERS^ 


TASK 

TRIAL  1 

TRIAL  2 

TRIAL  3 

1. 

"Enable  Fire" 

0/6357 

0/6357 

0/6357 

2. 

"Auto  Search"  Start 

20/4500 

20/4500 

20/4500 

3. 

Zoom  Seeker  Manually  (1st) 

35/2942 

17/4700 

33/2973 

4. 

Slew  Seeker  Manually  (1st) 

28/3610 

18/4600 

23/3821 

5. 

Target  In  Field  Of  View 

7/5778 

6/5819 

24/3768 

6. 

Start  Seeker  Slew  To 

Target  (Target  Detected) 

31/3289 

17/4700 

32/3079 

7. 

End  Seeker  Slew 

48/1332 

35/2800 

48/1400 

8. 

First  "Lock-On"  Try 

48/1332 

38/2512 

49/1359 

9. 

First  "Break-Lock" 

49/1280 

39/2457 

49/1359 

10. 

Second  "Lock-On"  Try 

None 

57/1564 

None 

11. 

Second  "Break-lock" 

— 

58/1525 

— 

12. 

Target  Identified 

Don't  Kxw 

Don't  Yikm 

Don't  Kiow' 

13. 

Missile  Reaches  Target  Area 

1: 04/Hit 

1: 06/Miss 

59/Miss 

14.  Trial  1  Conments:  Gunner  used  manual  guidarx%  to  inpact  after  1st  break- 
lock  oocured. 


]5.  Trial  2  Conments;  Missed,  gunner  pegged  seeker  slew  lookii>g  to  the  right, 
this  ^o*^d  be  the  reason  vAiy  he  flew  to  the  left  of  the  target,  didn't  point 
the  missile  to  v4iere  he  was  looking. 


16.  Trial  3  Conments:  Brr’  ?  lock  at  49  seconds,  flew  by  target  at  59  seconds, 
crashed  vMle  making  a  manual  turn  at  1:18. 
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TABLE  4 


ENGAGEMENT  TASK  PERFt»MANCE  CUMUIATIVE  TIME,  AND  RANGE  TO  TARGET  AREA  - 
SCENARIO  #18  (Two  Helicxpters  Turning  Ttward  Fire  Unit) 


TTrACTMPNT  TIME rSEODNDS^ /RANGE  TO  TARGET  AREA  (METERS) 


TASK 

TRIAL  1 

TRIAL  2 

TRIAL  3 

1. 

"Enable  Fire" 

0/8302 

0/8302 

0/8302 

2. 

"Auto  Secirch"  St2u± 

38/4487 

38/4487 

38/4487 

3. 

Zoom  Seeker  Manucdly 

(1st) 

26/5800 

49/3327 

55/2978 

4. 

Slew  Seeker  Manually 

(1st) 

15/7173 

44/3700 

14/7171 

5. 

Target  In  Field  Of  View 

28/7000 

56/2429 

48/3376 

6. 

Start  Seeker  Slew  To 

Target  (Target  Detected) 

20/6500 

56/2429 

53/2700 

7. 

End  Seeker  Slew 

47/3548 

56/2429 

1:13/* 

8. 

First  "lock-On"  Try 

48/3496 

1:54/* 

1:51/* 

9. 

First  "Break-Lock" 

59/2214 

1:55/* 

None 

10. 

Second  "Lock-On"  Try 

1:09/1212 

2:02/* 

— 

11. 

Second  "Break-lock" 

1:12/840 

None 

— 

12. 

Target  Identified 

Don't  Knew 

Don't  Knew 

Don't  Knew 

13. 

Missile  Reaches  Target  Area 

2: 21/Miss 

2: 08/Hit 

2 : 10/Miss 

14.  Trial  1  Octnments:  Broke  lock  abcut  5  times,  last  was  too  late  for  any 
other  actions.  Tried  to  lock-on  seeker  feir  out,  or  in  wide  field  of  view. 


15.  Tried  2  Cemments:  None 


16.  Trial  3  Cornments;  Good  early  detect,  waited  to  put  tracker  box  on  target, 
good  lock  xjntil  missile  flew  by  target  at  2:10. 


♦Missile  reached  original  target  area  and  distance  to  target  indicator  stopped 
at  499. 
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nCOREl 


RANGE  (KM)  7 
TIME  (SEC)  70 


nGURE2 


MICROSAINr  TASK  PERFORMANCE  HMELINE 


0.0  50.0  100.0  -150.0 

+  +  +  + 

aOOla  select  menu  option 

1  Prepare  to  Engage 

2  Rotate  OP  PHASE  knob  — 

2a  Verify  Fire  Unit  Pos 

2b  Write  Target  Coords  - 

3  Hook  Target 

3a  Select  Target  Type  -  . 

3b  Air  Target 

4  Load  Missile  Types 

5  Load  Launch  Interval 
5b  Accept  Route 

6q  Announce  Firel 
6a  Presses  FIRE  switch 

6b  Monitors  countdown  - 

6c  Missle  Elevates 

6d  Launch  Delay 

8q  Boost  Motor  On 
9q  Pitch  Over 

lOq  Missile  Flyout  - / 

12  No  FOV  Target 

14  zoom  in  - 

11  FOV  Target 

13  slew  seeker 
15aa  Detect  Target (s) 

15ab  Slew  Grab  Box 
15ac  Press  Grab  Switch 
15ad  Identify  Target 
15ae  Press  KEEP  switch 
15af  Offset  Aim 

90  Impact  Target 

+  +  +  + 

0.0  50.0  100.0  150.0 
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nGl]RE3 


NLOS  EMBEDDED  TRAINER  CHARACIERISnCS  EVALUATION 


1.  Time  required  to  transition  into  and  cwt  of  ET. 

-  Gunners  can  quickly  transition  into  and  out  of  ET. 


2.  ET  logistics  burden. 

-  No  ET  logistics  concerns  were  identified. 


3.  Levels  and  types  of  training  delivered  by  ET. 

-  Concern:  Content  is  limited  to  target  engagement,  could  be  broadened  in 
scope  to  optimize  totzd  PCI  ET  sustainment  training  potential. 


4.  Nuntoer  of  training  scenarios  and  variety  prevent  memorization. 

-  "Variable"  scenarios  provide  flexibility  needed  to  prevent  memorization. 


5.  Training  content  or  tcpics. 

-  Concern:  ET  automat iczdly  provides  target  intercept  point,  transparent 
to  the  gunner.  Artificial  feature  r^laces  key  mentzil  task  training. 


6.  Dependence  on  instructors  or  training  managers  when  vising  ET. 

-  Gunners  exercise  ET  without  assistance,  r^lay  with  on-board  videotape. 

7.  Required  level  of  fidelity  of  simulations. 

-  Ocnstraints:  Single  missile,  dayli^t  only,  no  target  identification, 
no  time  pressure  to  launch,  autonatic  target  intercept  cadculation. 

8.  Performance  evaluation  and  assessment  capability. 

-  Concern:  No  performance  sunmary  or  record  excqpt  videotape.  No 
diagnostic  conparison  of  ta.sk  performance  to  standards. 

9.  Performance  data  security. 

-  Ihe  only  potential  data  source  is  the  optional  videotape  record. 


10.  Feedback  in  support  of  effective  learning. 

-  Concern:  No  performance  feedback  against  an  "expert"  standard. 


11.  ET  adaptabillity,  respcnd  to  soldier  performance  and  strenghten  weak 
areas. 

-  Concern:  Need  to  evaluate  performance  and  provide  a  software  message  to 
direct  gunner  to  appropriate  ET  scenario  or  lesson  in  a  training  matrix. 
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rharacteristic  1.  Allowable  times  to  transition  into  and  out  of  ET. 

Overview:  Gunners  can  quickly  transition  into  and  out  of  the  ET  mode  of  operation. 

Observations: 

-  Time  required  to  transition  into  ET  scenario  menu  after  system  initialization  was 
^proximately  1  minute  and  30  seconds  for  the  first  scenario.  Missile  BIT  was  not 
required  between  ET  scenarios  so  that  the  time  between  the  end  of  one  scenario  and  the 
arrival  at  the  menu  for  the  next  scenario  was  qiproximately  1  minute.  Time  required 
from  the  selection  of  a  scenario  in  the  menu,  until  the  gunner  hits  the  "Fire  Em^le" 
switch  was  approximately  1  minute  and  20  seconds.  All  times  were  taken  from  the  pilot 
data  collection  effort 

>  Hme  required  to  transition  out  of  ET  is  minimal-  At  ai^  time  the  NLOS  gunner 
can  move  the  Gunner's  Console  "Training/FiringT  switch  into  the  Tiring"  position  and 
return  immediately  to  the  main  menu. 


Characteristic  2.  Allowable  logistics  burden  for  ET  and  any  ^pended  component 
requirement 

Overview:  The  NLOS  ET  system  logistics  support  requirement  does  not  appear  to  be 
different  from  that  of  the  fire  unit 

Observations: 

-  NLOS  ET  conqwnents  are  essentially  software.  The  NLOS  lOE  fire  unit  was  able 
to  hook-up  to  a  fixed  power  supply  to  eliminate  the  need  for  running  the  vehicle  motor. 


Characteristic  3.  Levels  or  types  of  training  to  be  delivered  by  Embedded  Training 
(e.g.,  acquisition,  sustainment,  e}q)ert). 

Overview:  Training  content  selection  should  begin  with  a  review  of  the  training  POL 
Try  to  expand  training  content  to  maximize  the  potential  of  ET  as  an  interactive 
conqniter  based  framing  system. 

Obser^tions: 

-  The  NLOS  lOE  ET  system  provided  target  engagement  training  at  the  skills 
sustairunent  level  The  ET  system  provided  switchology  familiarization,  and  practice  on 
NLOS-spedfic  hand-eye  coordination  tasks.  Target  identification  was  not  trained  due  to 
limitations  in  the  fidelity  of  the  screen  imagery. 
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-  The  training  content  of  the  NLOS  lOE  ET  system  may  be  too  limited,  as  it  trains 
engagement  tasks.  Future  NLOS  ET  development  should  work  toward  greater 
utilization  of  the  Computer  Assisted  Training  potential  of  the  on-board  interactive  video 
conqtuter  technology.  Much  of  the  NLOS  training  program  that  is  presented  in  the 
classroom  or  in  textbook  form  (to  include  the  line  drawing  and  still  picture  portions  of 
Aircraft  Identification  training)  could  be  sustained  with  ET.  A  first  step  would  be  to 
review  the  NLOS  training  Program  Of  Instruction  (POI)  identifying  those  tasks  which 
should  be  considered  for  ET  sustainment  The  ET  training  focus  should  be  on  critical 
tasks,  tasks  identified  as  needing  frequent  sustainment  training,  and  tasks  which  are  not 
commonly  practiced,  such  as  degraded  modes  of  operation,  and  maintenance  tasks. 


Characteristic  4.  Number  of  scenarios  and  the  required  variability  of  training  (e.g., 
location-specific  scenarios,  expert-level  scenarios).  Also,  sufficient  variety  to  prevent 
memorization  and  boredom. 

Overview:  The  NLOS  lOE  ET  system  provides  26  "Fixed"  and  16  "Variable"  target 
engagement  scenarios.  Some  of  the  NLOS  gunners  appeared  to  have  memorized  key 
features  of  the  Fixed  scenarios,  such  as  where  to  look  for  targets,  and  how  targets  would 
move.  In  contrast,  the  Variable  scenario  option  appears  to  provide  for  sufficient  variety 
to  prevent  memorization  and  boredom,  however  problems  encotmtered  in  collecting  data 
on  Variable  scenarios  limits  this  conclusion. 

Observations: 

-  The  ET  database  contains  sixteen  different  simulated  battlefield  scenarios.  Each 
scenario  contains  one  or  more  targets  to  include  helicopters,  tanks,  trucks,  jeeps, 
ammunition,  or  buildings. 

-  For  Fixed  scenarios  the  sixteen  basic  scenarios  are  rearranged  to  present  twenty-six 
canned  scenarios  where  fire  unit  location  and  target  movement  are  predetermined.  The 
same  targets  will  always  appear  in  a  given  scenario,  and  for  moving  targets  an  intercept 
area  is  provided  to  the  gunner.  It  was  observed  that  some  gunners  had  memorized  some 
aspects  of  the  Fixed  scenarios  so  that  they  could  search  in  a  limited  area  where 

they  knew  the  targets  would  be. 

-  For  Variable  scenarios  the  gunner  is  presented  with  the  sixteen  basic  scenarios,  but 
has  the  option  of  selecting  alternative  sites  to  position  the  fire  unit  at  and  engage  targets, 
thus  allowing  the  gunner  to  build  a  number  of  different  engagements  around  the  basic 
set  of  sixteen  target  scenarios.  Variable  scenarios  present  a  random  sub-set  of  targets 
from  a  larger  pool  for  each  scenario  which  should  add  to  variety  and  lessen  the  chance 
of  memorization. 

-  One  means  to  increase  variability  and  prevent  memorization  of  the  limited  number 
of  scenarios  would  be  to  simply  call  up  the  same  scenarios,  but  under  an  extended  menu 
numbering  scheme.  While  gunners  might  be  able  to  remember  basically  what  happens  in 
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16  or  26  scenarios,  th^  would  be  less  likely  to  do  this  if  each  scenario  were  repeated  in 
random  order  four  times  so  that  perh^  100  scenario  titles  were  presented  in  the  menu, 
each  with  a  slightly  different  description  or  set  of  target  data. 


Characteristic  S.  Training  content  or  topics  (e.g.,  failure  modes,  multiple  targets, 
maintenance). 

Overview:  A  discussion  of  the  adequa<y  of  NLOS  ET  content  or  topics  should  begin 
with  the  identification  of  the  total  NLOS  training  program  needs,  perhaps  best  reflected 
in  the  training  POL  The  NLOS  lOE  ET  system  training  content  was  limited  to  target 
engagement  tasks.  ET  content  should  be  expanded  to  optimize  its  potential  for 
sustainment  of  other  training  content,  perha^  with  a  greater  emphasis  on  sustainment  of 
classroom  and  textbook  material  presented  in  the  POL 

Observations: 

-  One  major  concern  is  that  the  ET  system  artificially  performed  a  task  that  the 
gunners  need  to  be  trained  to  perform.  The  system  automatically  calculates  an  intercept 
point  for  moving  targets,  transparent  to  the  gunner,  and  presents  this  to  the  gunner 
instead  of  the  original  target  coordinates,  so  that  the  mi^e  will  automatically  fly  into 
the  path  of  the  target  In  real  life  the  target  intercept  calculation  task  will  probably  be 
allocated  to  the  gunner,  not  the  machine.  The  ourent  ET  ^tem  may  be  doing  a 
disservice  to  the  gunner  by  masking-over  this  important  training  requirement 

•  A  niunber  of  training  elements  were  not  included  in  the  ET  ^tem: 

■  multi-round  launch  where  the  gunner  must  assume  control  of  a  second  missile 
already  late  in  its  flight 

-  recognition  and  identification  tasks  were  not  exercised,  which  makes  extra  time 
available  for  the  gunner  to  perform  other  time  stressed  end  game  tasks. 

-  the  target  movement  related  time  stress  for  planning  is  not  accurately  represented, 
as  targets  don’t  start  moving  until  the  Tire  Enable”  switch  is  activated.  Target 
engagement  task  performance  demands  would  be  better  represented  if  target  movement 
began  when  the  scenario  was  selected  from  the  training  menu. 

-  no  system  failure  modes. 

-  no  Imaging  Infrared  (ER)  seeker  scenarios. 

-  no  weather  or  climate  condition  variation. 
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Characteristic  6.  Allowable  extent  of  dependence  on  instructors  or  training  managers 
during  Embedded  Training  sessions. 

Overview:  ET  should  provide  a  stand  alone  training  capability  for  gunners.  To  the 
extent  that  an  NLOS  guimer  can  not  infer  what  his  mistakes  are,  and  what  corrective 
actions  to  take,  instructors  and  training  managers  will  be  needed  to  support  effective 
learning. 

Observations: 

-  Guimers  can  play  back  their  performance  for  assessment  using  the  on-board  video 
recorder. 

-  During  the  data  collection  recording  of  the  Variable  scenarios  it  was  found  that  the 
procedures  for  running  the  scenarios  were  not  well  understood  and  that  an  "operating 
instructions”  capability  was  not  built  into  the  system.  Consider  adding  a  "Help"  option  on 
the  menu  to  bring  up  a  page  of  instructions  on  how  to  set  up  and  proceed  throu^  the 
Variable  scenarios. 

-  Currently  no  automated  capability  for  student  task  performance  scoring.  Training 
managers  should  identify  key  tasks  and  create  a  performance  checklist  that  could  be  built 
into  the  ET  software  for  each  scenario  to  help  assess  individual  gunner  proficiency,  and 
platoon  training  needs. 

•  There  is  no  guidance  provided  within  the  ET  ^tem  for  scenario  sequencing.  A 
recommended  sequence  strategy  to  support  or  free-up  trainers  should  be  developed.  ET 
should  be  able  to  automatically  make  a  "next  scenario"  recommendation  (or 
requirement)  based  on  a  gunner’s  success  or  failure  in  a  given  scenario,  to  lead  the 
gunner  through  a  series  of  progressively  more  difficult  training  exercises. 

Characteristic  7.  Required  level  of  fidelity  of  simulations. 

Key  soldier  tasks  should  be  simulated  in  as  much  detail  as  possible  (e.g.,  visual  scene 
fidelity,  realistic  communications,  and  task  performance  time  constraints). 

Overview:  The  NLOS  visual  simulation  requirements  have  been  specified  in  terms  of 
required  target  image  size  for  a  given  range,  and  resolution  lines  on  target,  to  allow  for 
ret^tic  target  detection  training.  NLOS  engagement  training  requirements  include 
communications  (call  for  fire),  mission  planning  (moving  target  intercept  calculation), 
missile  fly-out  and  search  phase,  and  the  target  engagement  endgame.  The  NLOS  lOE 
ET  system  shows  some  important  shortcomings  in  terms  of  communications  and  mission 
planning  that  could  be  corrected.  The  target  information  presented  in  the  scenario 
message  does  not  reflect  the  type  of  command  and  control  information  that  would 
normally  be  provided  to  the  gunner.  The  time  pressure  pushing  task  performance  in  a 
real  engagement  is  missing  because  target  movement  does  not  start  imtil  the  missile  is 
launched  (this  could  be  changed  to  the  time  the  scenario  is  selected  from  the  menu). 


The  important  task  of  calculating  a  missile  intercept  point  against  a  moving  target  is  not 
trained  in  the  Fixed  Mission  scenarios,  as  this  is  automatically  provided  to  the  gunner.  It 
remains  to  be  determined  whether  the  target  detection  range  performance  displayed  by 
gunners  (targets  detected  in  the  seeker  field  of  view  (FOV)  as  far  out  as  7km  in  scenario 
#18)  is  an  accurate  reflection  of  a  real  world  cs^ability. 

Observations: 

-  The  scenario  menu  target  information  does  not  match  what  would  be  expected  in  an 
air  defense  call  for  fire.  T^  should  be  an  easy  situation  to  correct  As  example,  for 
Fixed  scenario  #16  the  ET  menu  tells  the  gunner  "A  4  HELD  SQUADRON".  Based  on 
the  hard  copy  supporting  information  for  this  scenario  the  gunner  could  be  told  "Hinds, 
four,  position  coordinate  is  391372,  moving  on  azimuth  280  degrees  at  70  meters  a 
second". 

-  The  ET  engagements  against  moving  air  targets  significantly  imderestimate  some 
key  mental  task  requirements.  It  is  inqxirtant  to  recognize  that  the  NLOS  ET  system 
automatically  calculates  a  target  intercept  point  designed  to  take  the  missile  into  the 
path  of  the  target  •  it  does  not  present  ^e  initial  position  of  a  moving  air  target 
Likewise,  the  ET  system  does  not  start  target  movement  until  the  moment  when  Tire 
Enable"  occurs,  so  that  the  target  does  not  move,  and  the  intercept  point  does  not 
change,  while  the  gunner  prepares  to  fire.  In  a  real  engagement  the  gunner  will  have  to 
be  able  to  calculate  the  moving  target  intercept  point,  and  will  have  to  be  able  to  factor 
in  any  delays  that  occur  after  the  original  target  location  is  identified,  to  include  the  time 
required  to  make  the  intercept  calculation.  ET  target  areas  which  represent  stationary 
ground  targets  are  the  approximate  initial  position  (and  terminal  position)  of  the  target 
or  a  group  of  targets. 

-  Target  area  entry  and  update  was  not  required  in  the  26  Fixed  scenarios. 

-  Launch  tasks  iq)pear  to  be  an  accurate  representation  of  real  tasks. 

-  Missile  fly-out,  and  auto  search  to  bring  a  target  into  the  seeker  FOV  appears  to 
provide  an  accurate  representation  of  mechanical  tasks. 

-  Is  the  ET  system  target  image  size  an  accurate  representation  of  the  image  size  a 
guimer  would  see  on  the  screen  in  a  real  engagement?  Look  at  detection  range  data, 
comparing  ET  performance  to  Oq)tive  Flight  Testing  and  Live  Fire  detection  range  data 
to  establish  the  validity  of  performance  trends  obtained  from  the  ET  system. 

-  Given  detection,  target  identification  is  a  significant  and  difficult  task  that  should 
require  some  time  in  the  final  moments  of  the  engagement  Target  ID  was  not  played  in 
NLOS  lOE  ET  trials  that  were  videoUq>ed,  as  all  detected  targets  were  engaged.  The 
target  identification  task  and  time  requirement  should  be  modeled  for  further  evaluation. 
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-  The  NLOS  ET  system  makes  no  provision  for  multiple  missile  launch  and  the 
resulting  time  stressed  end  game  tasks. 

-  Missiles  frequently  appeared  to  veer  off  both  ground  and  air  targets  at  the  last 
second  prior  to  impact 

•  Seeker  break-lock  occurred  frequently  during  Embedded  Training  scenarios.  Is  the 
ET  seeker  performance  an  accurate  reflection  of  real  seeker  performance? 

-  UR  seeker  engagements  are  not  simulated  with  the  ET  system. 

-  The  NLOS  lOE  ET  system  allowed  gunners  to  fly  missiles  in  a  180  degree  turn  and 
come  back  at  a  target  that  has  been  missed.  This  missile  c£q)ability  has  not  been 
demonstrated.  Should  gunners  train  for  a  ’TJ-Ttun”  capability  with  the  ET  system? 


Characteristic  8.  Performance  evaluation  and  assessment  capability. 

Overview:  ET  should  provide  for  (1)  a  performance  assessment  capability,  (2) 
2q>propriate  feedback,  and  (3)  record  keeping.  The  NLOS  ET  system  does  not  provide 
these  features  as  an  automat^  function,  and  this  could  serve  to  limit  the  training 
effectiveness  of  the  system.  It  should  be  possible  to  have  these  three  functions 
automated  into  ET.  Assessment  and  feedback  must  be  built  into  the  system,  so  the  crew 
can  train  itself,  whidi  is  the  purpose  of  ET. 

Observations: 

-  The  NLOS  lOE  ET  tystem  allowed  gunners  to  go  through  the  process  of  a 
simulated  engagement,  and  to  replay  a  video  tape  of  the  trial. 

-  The  NLOS  lOE  ET  system  did  not  record  or  present  summary  task  performance  for 
each  scenario  played,  and  guimer  performance  was  not  referenced  against  any 
performance  standard. 

-  There  was  no  indication  of  whether  the  NLOS  gunner  had  engaged  a  friend  or  a  foe 
(fratricide).  This  information  might  easily  be  presented  as  a  simple  message  to  the 
guimer  at  the  conclusion  of  scenario  play,  "Target  k  AH-64,  Do  Not  Engage". 

-  The  NLOS  lOE  ET  system  displayed  a  great  deal  of  task  performance  information 
during  scenario  play  which  could  be  valuable  for  performance  assessment  and  feedback  if 
it  were  oqitured  and  summarized.  This  task  performance  information  included  a 
continuous  readout  of  missile  range  to  target  area,  which  also  allowed  for  an  estimate  of 
available  task  performance  time.  Switch  flip  actions,  seeker  zoom  and  gimbals  angle, 
seeker  lock  ai^  break-lock  performance,  and  missile  impact  time  might  also  be  recorded 
as  data  points  to  assess  whether  guimers  perform  tasks  in  the  order  and  time  prescribed 
by  training,  or  whether  they  are  making  errors  which  require  correction. 
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Perfonnance  data  security. 


Observatioiis: 

-  Currently  the  only  source  of  performance  data  would  be  the  on-board  recorder. 


Characteristic  10.  Feedback  in  support  of  effective  learning. 

Overview:  Topically*  ^th  regard  to  human  task  performance  "You  get  what  you 
measure".  If  ET  seeks  to  shape  gunner  behavior  toward  a  standard  of  task  performance, 
then  ET  must  feed  back  performance  information  on  the  specific  subset  of  tasks  in 
question.  The  current  ET  system  can  only  provide  a  replay  of  the  scenario,  no 
evaluation,  assessment,  summary  of  performance,  or  con:q>arison  to  any  "e)q>ert"  or  "right 
way"  performance  standard  for  the  scenario.  If  switch  flip  actions,  button  pushes,  or 
screen  data  can  be  captured  for  a  sununaty  screen  presentation  at  the  end  of  the 
scenario  exercise  NLOS  ET  feedback  might  be  provided  in  several  forms. 

Observations: 

-  By  simply  workmg  through  scenarios  the  gurmer’s  learning  is  dependent  on  his  short 
term  memory  (guimer  recall)  of  task  actions  and  performance  outcomes. 

-  By  using  the  video  replay  feature  the  gunner  can  refiesh  his  memory  of  actions  and 
outcomes,  but  if  be  can’t  recognize  what  went  wrong  this  review  may  not  help  him. 

-  A  simple  summary  record  of  key  switch  flip  type  tasks  and  when  they  occur  with 
regard  to  time  and/or  range  to  target  area  might  a  significant  inq>rovement  to 
support  effective  learning.  An  example  of  this  type  of  feedback  is  presented  as  Figure  4. 

-  A  more  accurate  representation  of  task  performance  demands  might  be  made  Ity 
identifying  "windows"  of  time  (or  range  to  target)  in  which  the  guimer  can  perform 
actions  without  adversefy  impacting  the  engagement  outcome.  The  "windows"  ^proach 
has  the  advantage  of  moving  away  from  the  single  right  answer  "Eiqiert"  solution,  and 
also  identifies  that  performing  a  task  "too  early"  might  be  as  bad  as  "too  late",  llie 
"windows"  would  be  identified  by  working  with  the  gunners  and  supporting  personnel  to 
identify  useful  or  needed  feedback,  and  work  to  automate  this  csqiability  as  much  as 
possible.  As  exanqile,  the  following  might  be  provided  as  performance  feedback: 

(1)  Broke  auto  search  too  early 

(2)  Seeker  zoom  magnification  too  wide  for  lock-on  attempt 

(3)  Attempted  seeker  lodi-on  too  far  from  target 

(4)  Fratricide  -  killed  a  friendly  target 

An  illustration  of  how  this  task  performance  information  and  action  feedback  might 
be  presented  to  a  gunner  is  provid^  as  Figure  5. 
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TASK  PERFORMANCE:  a)MPARISON  TO  STIANDARD 
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-  Target  intercept  prediction  accuracy.  ET  could  be  veiy  valuable  in  target  intercept 
prediction  training.  Replay  target  data,  the  intercept  formula,  and  the  correct  intercept 
solution  V.S.  the  student’s  predicted  intercept  target  area  input 

•  Again,  coverage  of  training  content  beyond  the  target  engagement  tasks  to 

the  broader  training  POL 


rharacteristic  11.  Adaptability,  so  that  the  system  can  respond  to  soldier 
performance  and  zidapt  the  presented  training  materials  to  strengthen  weak  areas. 

Overview:  In  order  to  respond  to  soldier  performance  there  must  be  some 
assessment  of  this  performance.  It  should  be  a  straight  forward  task  to  build  in  a  "go  to" 
recommendation  for  the  next  scenario  to  try,  ^ven  success  or  foilure  in  the  previous 
scenario  trial. 

Observations: 

-  Build  a  matrix  of  progressively  more  difficult  scenarios  which  center  around  basic 
skills  such  as  mission  planning,  target  identification,  flying  a  dog-leg  course,  and  others. 
Such  a  training  matrix  is  built  into  the  Ml  tank  Unit  Conduct  Of  Fire  Trainer. 

•  The  variable  scenarios  could  be  highly  ad^table  to  respond  to  soldier  training 
needs,  as  different  scenarios  can  be  produced  hy  sinq>ly  changing  the  initial  vehicle 
position  and  target  information  provided  to  the  guimer  in  the  scenario  menu. 


6.  DISCUSSION:  Overall,  the  NLOS  lOE  ET  ^tem  provided  a  fairly  realistic 
engagement  simulation  which  would  be  useful  in  sustaining  procedural  skills  and  hand- 
eye  coordination  tasks. 

-  Suggestions  short  list: 

-  Start  with  the  NLOS  POI  and  identify  all  training  content  that  could  be  sustained 
through  ET,  not  just  the  engagement 

-  Identify  some  task  performance  standards,  time  or  range  to  target  referenced 
criteria  for  desired  performance. 

-  As  a  minimum,  add  some  basic  switch  flip  task  time  and  range  to  target 
performance  feedback  for  the  gunners  at  the  conclusion  of  each  scenario. 

-  Present  the  scenario  menu  information  phrased  as  FAADS  command  and  control 
messages. 
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-  Make  a  provision  for  target  movement  to  start  when  the  scenario  is  first  selected 
fi-om  the  menu,  so  that  the  time  sensitive  pre-laundi  tasks  are  realistically  trained. 

-  Train  the  target  intercept  calculation  task. 


7.  CONCLUSION:  The  NLOS  lOE  ET  system  sy>peared  to  be  a  valuable  tool  in  its 
present  form  for  providing  target  engagement  training  at  the  skills  sustainment  level 
The  ET  ^tem  provided  switchology  familiarization,  and  practice  on  NLOS-specific 
hand-eye  coordination  tasks.  Target  identification  was  not  trained  due  to  limitations  in 
the  fidelity  of  the  screen  imageiy.  The  NLOS  lOE  ET  system  fails  to  provide 
performance  feedback  to  the  gunner  which  is  a  significant  concern.  A  systematic  review 
of  gunner  task  performance  trends  can  provide  performance  standards  diat  might  be 
built  into  the  system  for  an  enhanc^  training  potential 


End  WILLIAM  SANDERS 

as  Research  P^chologist 
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APPENDIX  1 


ET  TARGET  MOVEMENT  DATA  AND  ENGAGEMENT  MAP 

FIXED  MISSION  I  16 


MENU  DESCRIPTION:  A  4-HELICOPTER  SQUADRON 
TARGET  DEFINITION:  4  SOVIET  HELICOPTERS  (HIND) 

NARRATIVE  DESCRIPTION: 

A  4 -HELICOPTER  SQUADRON  -  AS  YOU  ARE  COMING  OVER  THE  HILL  ON 
YOUR  LEFT,  THE  HELICOPTERS  SHOULD  BE  VISIBLE  IN  THE  DISTANCE.  IF 
THEY  ARE  COMING  FROM  THE  LEFT,  THEY  WILL  CHANGE  DIRECTION 
SLIGHTLY  AND  HEAD  TOWARDS  YOU.  AS  THEY  COME  NEARER,  THEY  WILL 
CONTINUE  TO  CHANGE  DIRECTION  (TURNING  TO  YOUR  LEFT)  AND  WILL  ALSO 
SPEED  UP. 


MISSION 

INFORMATION: 

DMG 

VAR. 

MISSN. 

LS 

INTERCEPT 

SCEN# 

TRACK# 

OFFSET 

COORD: 

COORD: 

9 

903 

8 

383200E 

388750E 

3727800N 

3724700N 

TARGET 

TIME 

HISTORY 

INFORMATION 

(NORMALIZED  FOR  8  SECOND  INTERVALS)  : 


X-DIS 

Y-DIS 

VEL 

(M) 

(M) 

(M/S) 

0 

SECONDS ;  0.0 

0.0 

0.0 

8 

SECONDS;  -68.4 

12.1 

69.4 

16 

SECONDS;  -68.4 

12.1 

69.4 

24 

SECONDS;  -68.4 

12.1 

69.4 

32 

SECONDS:  -68.4 

12.1 

69.4 

40 

SECONDS:  -61.2 

27.9 

67.2 

48 

SECONDS;  -53.9 

43.7 

69.4 

56 

SECONDS:  -42.2 

49.5 

65.1 

64 

SECONDS ;  -7.0 

66.9 

67.3 

72 

SECONDS ;  -7.0 

66.9 

67.3 

80 

SECONDS ;  -7.0 

66.9 

67.3 

88 

SECONDS;  -17.2 

24.1 

29.6 

96 

SECONDS;  -17.2 

24.1 

29.6 

HEADING  POSITION 

(DEGREES)  (UTM  METERS) 


0.0 

392230E 

3723570N 

280.0 

391683E 

3723667N 

280.0 

391136E 

3723763N 

280.0 

390589E 

3723860N 

280.0 

390042E 

3723956N 

294.5 

389552E 

3724179N 

309.0 

389121E 

3724529N 

319.6 

388783E 

3724925N 

354.0 

388727E 

3725461N 

354.0 

368670E 

3725996N 

354.0 

388614E 

3726532N 

324.5 

388476E 

3726725N 

324.5 

368338E 

3726918N 

AVERAGE  VELOCITY  -  61.7 
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appendix  1  (continued) 


FIXED  MISSION  #16 
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APPENDIX  2 


PILOT  STUDY  TARGET  ENGAGEMENT  TASK  UST 


1. 


2. 


3. 


4  . 


5. 


6. 


7. 

8. 


9. 


10. 


11. 


Select  "Emplacement''  from  main  menu. 

Select  "ENGINE  ON". 

Select  "MISSLE  BIT" 

Select  "MAIN  MENU". 

Select  "SIM  LAUNCH". 

Select  "FIXED"  or  "VARIABLE"  ET  engagement  scenario. 

Select  scenario  from  menu  (pull  trigger  to  second  detent) . 
Select  "ENGINE  ON". 

Select  number  of  missiles  (always  one) . 

Hook  target  (place  icon  on  target  box  using  joystick  and 
pull  through  to  second  detent),  or  type  in  coordinates. 

Computer  asks  if  gunner  wants  to  change  any  information 
(gunner  selects  "YES"  or  "NO") . 


12.  Select  "FIRE  ENABLE". 

(BOOST  MOTOR  ON,  MISSILE  PITCH  OVER,  MISSILE  FLYOUT  BEGINS) 

13.  Gunner  performs  missile  azimuth  correction  if  necessary. 
Gunner  compares  screen  imagery  to  memory  of  map  features  to 
determine  if  missile  is  flying  on  correct  asimuth.  If 
azimuth  correction  is  necessary,  gunner  moves  the  joystick 
to  slew  the  missile  seeker  until  the  desired  flight  path  is 
reached,  then  pulls  the  joystick  trigger  to  second  detent  to 
input  the  new  azimuth.  Not  required  with  the  ET  scenarios. 

14.  Select  RIGHT  TURN  or  LEFT  TURN  to  change  missile  flight 
direction  90  degrees. 

15.  Rotate  "BRIGHTNESS"  adjustment  knob  as  required. 

16.  Rotate  "CONTRAST"  adjustment  knob  as  required. 
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APPENDIX  2  (continued) 


(••auto  SEARCH”  SEEKER  ZOOM  AND  SLEWING  BEGINS  APPROXIMATELY  4 . 5KM 

FROM  TARGET  AREA) 

17.  Slew  missile  seeker  manually  by  pressing  the  joystick  thumb 
control  in  the  desired  direction. 

18.  Gunner  can  manually  adjust  the  seeker  zoom  magnification  by 
pressing  the  "ZOOM  IN”  or  "ZOOM  OUT”  pushbuttons. 

19.  Gunner  monitors  "RANGE  TO  TARGET"  distance  screen  readout  to 
anticipate  when  targets  might  appear  on  the  screen. 

20.  Gunner  visually  detects  probable  target,  assumes  manual 
seeker  control  if  in  the  "AUTO  SEARCH"  mode. 

21.  Gunner  visually  investigates  probable  target,  moves  seeker 
crosshairs  over  target  and  zooms  seeker  in  if  necessary. 

22.  Gunner  visually  identifies  the  target  as  a  threat  or  friend. 

23.  Gunner  determines  when  missile  is  close  enough  to  the 
target,  and  when  target  provides  sufficient  contrast  against 
the  background,  and  attempts  a  seeker  lock-on. 

24.  Gunner  attempts  lock-on  by  slewing  the  seeker  crosshairs 
onto  the  target  and  pressing  the  "TGT  DES"  pushbutton. 

25.  If  gunner  detects  that  the  seeker  has  broken  its  contrast 
lock  on  the  target  he  presses  the  "TGT  REJ"  pushbutton. 

26.  Gunner  repeats  the  lock-on  procedure,  or  manually  guides  the 
missile  to  impact  by  keeping  the  crosshairs  on  the  target. 

27.  Given  seeker  lock-on,  the  gunner  can  further  lock  the 
missile  onto  a  particular  area  of  the  target  by  placing  the 
crosshairs  on  the  target  impact  point  and  pressing  the 
"OFFSET  AIMPT"  button. 

28.  Gunner  visually  determines  missile  impact  on  the  target  or 
miss. 

27.  Gunner  can  select  two  RIGHT  or  LEFT  turns  to  make  a  180 
degree  turn  and  attempt  to  relocate  a  missed  target. 

28.  After  missile  impact,  gunner  flips  the  "FIRE  ENABLE"  switch 
down. 

29.  Select  "ESCAPE",  and  then  "SPARE  1"  pushbuttons. 

30.  Select  "RETRACT  MISSILE"  pushbutton. 

31.  Computer  is  now  back  in  the  system  menu. 
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APPENDIX  3 


NLOS  ENGAGEMENTTASK  TIME  AND  RANGE  VBDBO  REDUCDON  SHEET 


1.  start  Engine  message  (push  PDP  button) 

2 .  Select  #  missiles  message  (push  PDP  button) 

3.  Hook  Target  message  (joystick,  keyboard,  or  azimuth) 

4.  Change  Target  Info  message  (push  button  for  yes  or  no) 

5.  If  yes  on  5  above,  change  target  info  (how?) 

6.  Push  Enable  Fire  switch 

7.  Target  range  at  zero  seconds 

8.  Auto  search  starts 

9.  Manual  over  ride  of  auto  search 

10.  Zoom  seeker  manually 

11.  Slew  seeker  manually 

12.  Target  in  FOV 

13.  Target  detected,  gunner  starts  seeker  slew 

14.  Gunner  ends  seeker  slew  onto  target 

15.  First  lock  attempt 

16.  First  Tracker  Coasting  message 


APPENDIX  3  (continued) 


17.  Second  lock  attempt 

18.  Second  Tracker  Coasting  message 

19.  Target  ID 

20.  Missile  impact 


c 
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APPENDIX  D 

NLOS  EUE  TASK  ILLUMINATION  EVALUATION 


DEPARTMENT  OF  THE  ARMY 


U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Fort  Bliss  Field  Unit,  P.O.  Box  6057 
Fort  Bliss,  Texas  79906^7 


PERI-SBC  (70-n) 

26  September  19M 

MEMORANDUM  THROUGH  DIR,  ARI  FAAD  MANPRINT  TASK  FORCE  (ATTN: 
DR  DEPONTBRIAND) 

TO:  TRADOC  SYSTEM  MANAGER  -  FAAD,  ATSA-TSM-F,  (ATTN: 
MAJFEDAKO) 

SUBJECT:  NLOS  EUE  TASK  ILLUMINATION  MANPRINT  EVALUATION 


1.  INTRODUCTION: 

a.  The  present  evaluation  investigated  whether  NLOS  gunners  could  perform  tasks 
imder  conditions  of  light  discipline,  whether  light  attachment  points  could  be  identified 
that  would  free  the  gunner’s  hands,  and  provided  a  "quick  look"  comparison  of 
alternative  illumination  sources  which  might  be  used  to  reduce  the  NLOS  light  signiture. 
Concerns  regarding  NLOS  task  illumination  requirements  were  identified  in  the  NLOS 
Initial  Operational  Evaluation  (lOE)  system  Force  Development  Test  and 
E]q)erimentation  (FDTE)  Test  Report,  and  in  follow-on  discussions  within  the  NLOS 
Human  Engineering  Working  Group  (HEWG).  The  use  of  unfiltered  flashlight, 
chemical  light  sticks,  and  other  vehicle  light  sources  was  not  controlled  (light  discipline) 
during  NLOS  FDTE  which  may  have  significantly  altered  the  tasks  that  were  performed 
and  influenced  task  performance  time  and  accuracy.  The  simple  requirement  for  the 
crew  to  hold  a  flashlight  while  performing  in-cab  tasks  at  times  resulted  in  a  "one-armed" 
crewmember.  During  the  4  June,  1990,  HEWG  meeting  discussion  of  these  concerns, 
the  desire  to  e^lore  ways  to  minimize  NLOS  light  signiture  was  identified  as  an  issue 
for  e:q>loration.  The  sin^e  missile  night  reload  battle  drill  was  chosen  to  investigate  task 
illumination  issues,  though  no  time  standard  exists  for  single  missile  reload  itself.  Three 
task  illumination  issues  were  examined: 

Issue  1.  The  abiliQr  of  NLOS  gunners  to  perform  night  reload  tasks  under  conditions 
of  light  discipline  using  two  alternative  light  sources:  the  standard  issue  flashlight  with 
red  filter,  and  a  prototype  low  signature  cluster  map  light  (CML)  flashlight 

Issue  2.  The  relative  light  source  Red-Infi-ared  signature  of  three  alternative  task 
li^t  sources:  standard  issue  Amy  flashlight  with  a  Red  Filter,  standard  Army  flashlight 
with  a  prototype  Blue-Green  Filter,  and  a  prototype  CML  flashlight 
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Issue  3.  NLOS  gunner  recommendations  for  locating  light  holding  fixtures  to  allow 
for  hands-fi'ee  task  performance  at  night 


b.  The  NLOS  task  illumination  concern  was  chosen  for  evaluation  as  part  of  the 
NLOS  Extended  User  Employment  (EUE)  MANPRINT  program  as  a  means  of 
demonstrating  three  key  elements  of  the  MANPRINT  £q>proach:  (1)  early  and 
continuous  evaluation  of  soldier  issues  to  influence  system  design,  (2)  development  and 
2q>plication  of  new  tools,  and  methods  to  analyze  system  performance  issues,  and  (3) 
refinement  of  issues  and  criteria  for  future  evaluations.  The  NLOS  task  illumination 
evaluation  was  conducted  over  a  period  of  two  days  during  normal  daylight  duty  hours. 
Use  of  commonly  available  resources  allowed  the  evaluation  to  be  conducted  at  no  cost 


2.  METHOD: 

a.  Overview:  On  25  September,  1990,  eight  single  missile  reload  battle  drills  were 
carried  out  in  a  simulated  night  environment  in  building  5805  at  Ft  Bliss,  Texas.  Half 
the  drills  were  conducted  with  Red  Filter  flashlights,  and  half  were  conducted  with  CML 
flashlights.  While  the  decision  to  cany  out  onfy  eight  trials  did  not  allow  for  statistical 
comparisons,  it  should  be  noted  that  only  two  night  MOPPO  single  missile  reload  battle 
drills  were  conducted  during  the  entire  course  of  NLOS  FDTE.  Two  crews  of  two 
gunners  each  carried  out  four  drills.  Crew  members  rotated  task  duties  after  each  trial. 
The  two  NLOS  crews  performed  the  single  missile  reload  battle  drill  tasks  identified  in 
BATTLE  DRILLS  FOR  NON-UNE-OF-SIGHT  FIBER  OPTICS  GUIDED  MISSILE 
SYSTEM  (NLOS  FOG-M)  Version  7  (18  May  1989)  (Attachment  1).  During  the  reload 
battle  drill  a  single  147  lb  dumn^  missile  was  removed  fi’om  and  reloaded  onto  the 
NLOS  system  launcher.  The  time  line  for  the  single  missile  reload  drill  was  measured 
fi’om  the  gunner’s  command  "Prepare  for  missOe  reload"  until  both  crewmen  completed 
all  tasks  and  returned  to  their  seats  in  the  crew  cab. 


b.  NLOS  Gunner  Demographic  Data:  The  evaluation  was  conducted  with  NLOS 
gunners  from  A  Battery,  2nd  Battalion,  6  Brigade  Air  Defense  Artillery  at  Ft.  Bliss, 
Texas. 


Crew  #1:  Guimers  1  and  2 
Crew  #2:  Gurmers  3  and  4 

Gunner  1:  MOS  16R,  Height  6’0" 
Gurmer  2:  MOS  16P,  Height  5’!!" 
Gunner  3:  MOS  16S,  Hei^t  5’8" 
Gurmer  4:  MOS  16R,  Height  5’ 10" 
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c.  Alternative  Light  Sources  Evaluated: 


1.  Standard  issue  flashlight  with  Red  Filter 

2.  Standard  issue  flashlight  with  a  prototype  Blue-Green  filter 

3.  Cluster  Map  Light  flashlight  This  light  incorporated  a  number  of  small  light 
sources  producing  li^t  in  a  limited  spectrum  to  achieve  its  illumination,  hence  the  name 
"cluster  light”.  This  prototype  light  was  somewhat  smaller  than  the  standard  issue 
flashlight  and  the  lens  head  is  in-line  with  the  body  of  the  flashlight  (see  figure  1).  In 
contrast  the  lens  head  of  the  standard  issue  flashlight  is  turned  90  degrees  from  the  body 
of  the  light  The 

CML  flashlight  has  an  on-off  switch  with  a  five-level  power  setting  at  its  base.  The  CML 
flashlight  had  a  mercury  switch  installed  so  that  in  one  setting  mode  the  light  would 
come  on  whenever  pointed  dowit 

4.  Ouster  Map  Light  "Clip  On"  light  This  prototype  light  used  the  multiple 
limited  spectrum  light  soinces  principal  as  does  the  CML  flashlight  The  light  was 
housed  in  a  circular  tube  approximately  four  inches  long  and  one  inch  wide,  and  pivoted 
within  a  clip-on  mounting  bracket  (see  figure  2).  The  light  was  powered  through  wire 
coimections  to  the  NLOS  24  volt  batteries.  The  CML  Clip-On  light  had  an  on-off  switch 
with  a  five-level  power  setting  at  its  base.  Specifications  appearing  on  the  light  were: 
Light,  Map,  Night  Vision  Conq)atible,  Model  LC-CL-3,  MRL  TEK-UTE,  INC,  Union 
Bridge,  MD,  21791.  28  Volt  D.C. 


d.  Video  Data  Recording  Equipment:  The  primary  source  of  data  collection  was  a 
Panasonic  AG-160  video  camera  fitted  with  the  Dark  leader  Model  3000  night  vision 
systenL  The  Dark  Invader  has  a  second  generation  intensifier  imaging  tube  and  provides 
ultra-low  light  passive  night  visioa  Typical  image  tube  resolution  is  36  line  pairs  per 
millimeter  across  the  image  tube’s  25  millimeter  fiber  optics  viewing  screetL 
Illumination  was  provided  by  two  SOO  watt  Infra-red  larrq>s.  Illumination  was  not  visible 
to  the  naked  eye,  but  was  siifficient  to  fully  illuminate  the  vehicle  and  reload  task 
performance.  Equipment  and  photo  technicians  were  provided  by  Ft  Bliss  TEXCOM 
Air  Defense  Artillery  Board  Instrumentation  Branch,  Photo  Sectiott 


e.  Pre-Test  Training:  On  24  September,  1990,  practice  drills  were  held  in  building 
5805  at  Ft  Bliss  to  familiarize  NLOS  gurmers  with  night  reload  tasks.  The  bay  was 
blacked-out  by  placing  cardboard  over  three  windows.  The  light  level  was  reduced  to  a 
point  where  no  objects  were  visible  at  first  (couldn’t  see  your  hand  in  front  of  your  face), 
though  some  night  vision  adaptation  occurred  during  each  battle  drill.  Between  each 
battle  drill  the  building  bay  door  was  opened  to  flood  the  room  with  bright  daylight  to 
reduce  any  night  vision  ad^tation  on  the  part  of  test  participants. 
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Figure  2.  Cluster  Map  Light  "Clip-On”  light. 


f.  Test  Conduct: 


1.  Issue  1:  Night  Reload  Battle  Drill  Performance  Evaluation.  To  facilitate  video 
recording,  the  outermost  missile  canister  on  the  drivers  side  of  the  vehicle  was  the 
missDe  removed  and  installed  for  all  drills  except  Trial  #3.  For  Trial  #3  the  outermost 
missile  canister  on  gunner’s  side  was  removed  and  installed,  to  present  a  different  task 
performance  perspective  to  the  camera.  After  the  eight  trials  were  completed  the  NLOS 
gunners  participated  in  a  structured  debriefing  using  the  questionnaire  in  Attachment  2. 
Order  of  the  drills  and  lighting  conditions  were  as  follows: 

Trial  #1:  Crew  #1,  GI  flashlight  (red  filter) 

Trial  #2:  Crew  #2,  GI  flashlight  (red  filter) 

Trial  #3:  Crew  #1,  Prototype  flashlight  (cluster  light) 

Trial  #4:  Crew  #2,  Prototype  flashlight  (cluster  light) 

Trial  #5:  Crew  #1,  GI  flashlight  (red  filter) 

Trial  #6:  Crew  #2,  GI  flashlight  (red  filter) 

Trial  #7:  Crew  #1,  Prototype  flashlight  (cluster  light) 

Trial  #8:  Crew  #2,  Protot^  flashlight  (cluster  light) 


2.  Issue  2:  Light  Signature  Evaluation:  The  Red  Filter  flashlight,  a  Blue-Green 
Filter  flashlight,  and  the  CML  flashlight  were  placed  on  the  concrete  floor  at  the  rear  of 
the  vehicle  approximately  fifty  feet  fi’om  the  camera.  The  three  lights  were  lined  up  five 
feet  £q>art,  facing  the  camera,  and  then  turned  forty-five  degrees  off  axis,  yielding  three 
clearly  defined  signature  images.  The  signatures  were  recorded  on  video  tape  using  a 
camera  with  a  night  vision  device  attached.  Comparing  the  size  of  the  light  images  on  a 
twenty-inch  diagonal  television  screen,  their  relative  signature  (area  of  bright  glare)  was 
determined. 


3.  Issue  3:  Light  Source  Location  Evaluation:  Each  crewmember  used  the  Red 
Filter,  Blue-Green  filter,  and  the  CML  Oip-On  and  flashlights  to  read  a  map  in  the 
NLOS  crew  cab  in  the  blacked  out  bay.  Crewmembers  identified  where  the  lights  could 
optimally  be  placed,  and  the  preferred  light  level  intensity  position  for  the  CML  Clip-On 
and  CML  flashlight  The  crewmembers  were  given  copies  of  a  structured  interview  guide 
to  familiarize  themselves  with  the  questions  that  would  be  asked  at  the  end  of  the  reload 
trials.  Some  data  was  gathered  on  cab  lighting  while  the  gunners  identified  optimal 
placement  and  illumination  levels. 
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3.  RESULTS: 


a.  Issue  1:  Task  performance  times  summary:  Battle  drill  performance  times  appear 
in  Table  1.  The  small  data  set  does  lend  itself  to  statistical  comparisons,  however 
several  observations  can  be  made.  The  crews  can  perform  reload  under  conditions  of 
light  discipline,  with  both  the  standard  issue  Red  Filter  flashlight  and  the  prototype  CML 
flashlight.  The  overall  mean  reload  time  was  7  minutes  and  44  seconds  for  the  eight 
trials.  Team  1  performed  the  reload  drill  on  the  average  of  1  minute  and  40  seconds 
quicker,  and  both  teams  performed  the  reload  drill  about  30  seconds  quicker  using  the 
standard  issue  Red  Filter  flashlight.  Performance  times  for  both  Red  Filter  and  CML 
flashlight  cannot  be  compared  to  any  FDTE  times,  as  no  record  of  single  missile  reload 
times  was  presented  in  the  FDTE  Test  Report  Also,  there  is  no  single  missile  reload 
performance  time  standard  in  the  NLOS  RCXl  Missile  canister  guide  alignment  was  the 
one  task  that  caused  delays  during  trials.  This  task  requires  lighting  for  precision 
alignment  of  a  ”U"  shaped  guide  at  the  front,  middle,  and  rear  on  the  bottom  of  the  147 
lb.  missile  canister  with  a  one-inch  square  rail  on  which  the  canister  rests. 


b.  Issue  2:  Light  Signature  Comparison:  The  video  taped  picture  of  the  Red  Filter, 
Blue-Green  filter,  and  CML  flashlight  signature  images  were  displayed  on  a  twenty-inch 
TV  and  measured  so  that  their  relative  signature  size  (area)  could  be  determined.  All 
measurements  are  approximate  as  the  image  sizes  were  small.  The  Blue  Filter  signature 
was  found  to  be  60.5%  of  that  of  the  Red  Filter  signature,  and  the  CML  flashlight 
signature  was  found  to  be  112%  of  that  of  the  Red  Filter  signature. 


c.  Issue  3:  Light  source  placement:  For  in-cab  tasks,  crewmembers  recommended 
placing  CML  Clip-On  lights  on  the  radio  rack,  one  on  each  side,  to  provide  lap  level 
illumination  for  driver  and  gunner.  No  requirement  was  identified  for  a  fixture  to  hold  a 
light  source  outside  of  the  cab  to  aid  reload  tasks.  From  a  review  of  the  video  taped 
trials  it  was  observed  that  at  times  crewmembers  would  simply  lay  their  flashlights  down 
on  the  vehicle  fenders  and  on  top  of  the  canisters  in  the  missile  rack  to  provide  task 
illumination.  Crewmembers  would  also  fasten  their  flashlights  on  their  web  gear. 
Members  of  crew  #1  typically  wore  the  CML  flashlight  dangling  from  a  cord  around 
their  neck.  The  task  of  aligning  the  three  missile  canister  guides  with  the  missile  rack 
support  rail  was  identified  as  requiring  a  great  deal  of  lighting  to  illuminate  the  fine 
alignment  work,  while  at  the  same  time  requiring  heavy  lifting  to  support  and  maneuver 
the  147  lb.  canister. 


4.  DISCUSSION 
a.  Evaluation  Issues: 
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Issue  1.  The  evaluation  revealed  that  NLOS  gunners  can  perform  reload  tasks  under 
conditions  of  light  discipline  with  both  the  standard  Red  Filtered  flashlight  and  the 
prototype  low  signature  CML  flashlight  Possible  future  action:  do  a  full  six-pack  reload 
against  the  RCX^  performance  standard. 

Issue  2.  Comparison  of  the  illumination  signatures  of  three  light  sources  showed  that 
in  comparison  to  the  Red  Filter  flashlight  a  Blue-Green  Filter  reduced  the  standard 
flashli^t  signature  to  about  60%.  The  signature  from  the  CML  flashlight  was  only 
about  11%  of  that  of  the  Red  Filter  flashlight  Possible  future  action:  do  a 
spectrometer  evaluation  of  the  alternative  light  sources  to  accurately  quantify  their 
relative  signatures,  and  redo  the  video  taped  recording  with  a  greater  magnification 
camera  lens  to  provide  a  larger  screen  image  of  the  light  sources. 

Issue  3.  NLOS  gunners  recommended  adding  CML  Clip-On  lights  in  the  area  of  the 
radio  rack  to  allow  for  lap  level  illumination  for  both  the  gunner  and  the  driver.  No 
light  holding  fixtures  were  identified  for  use  in  missile  reload  tasks.  Possible  future 
action:  can  we  provide  simple  brackets  on  the  radio  rack  where  both  crewmen  can  clip 
on  their  standard  issue  flas^ghts  for  hands-free  illumination  of  lap  top  tasks? 


b.  l^iANPRINT  Process  Demonstration:  NLOS  task  illumination  was  explored  as  a 
MANPRINT  concern  to  illustrate  the  process  of  follow-on  evaluation  of  lessons  learned 
from  testing,  identification  of  inexpensive  methods  and  tools  to  analyze  system 
perfom  lance,  and  the  identification  of  ways  in  which  to  improve  future  performance 
testing.  The  evaluation  followed  up  on  unanswered  questions  from  NLOS  FDTE,  and 
the  fine  ings  should  have  implications  for  the  design  of  the  FSD  system  reload  equipment. 
The  ev;  duation  was  conducted  in  part  to  show  the  utility  of  doing  low  cost  part-task 
motor  ]KX)1  evaluations.  Eight  MOPPO  night  reload  drills  were  conducted  during  the 
EUE  evaluation  compared  to  only  two  drills  conducted  during  NLOS  FDTE.  There  was 
no  cost  associated  with  the  EUE  evaluation  as  it  relied  on  commonly  available  facilities 
and  wa»  conducted  during  normal  duty  hours.  The  use  of  Night  Vision  equipped  video 
tape  eqjuipment  allowed  the  evaluation  to  be  conducted  with  only  one  dau  collector. 

jphasis  on  maintaining  night  discipline,  and  measurement  of  light  signature  should 
)rated  into  NLOS  FDTE  II  testing.  Inexpensive  motor  pool  night  battle  drill 
should  be  conducted  prior  to  going  into  any  major  operational  test  I>on't  put 
trering  simple  questions  until  a  big  test,  sooner  is  better. 


c.  Detailed  Discussion  Of  Trial  Task  Performance 

1.  Task  lighting  requirements:  Task  lighting  requirements  were  identified  from  a 
review  of  the  video  tape  for  the  eight  trials.  Soldiers  demonstrated  that  they  did  not 
need  any  light  source  to  remove  the  brush  guard.  Light  was  used  during  insertion  of  the 
reload  boom  pedestal  pin  and  to  find  the  chain  hoist  laying  on  the  floor  of  the  building. 
Light  was  again  used  for  the  task  of  positioning  the  chain  hoist  assembly  on  the  reload 
boom  raiL  Light  vras  used  in  attaching  the  sling  around  the  canister,  and  to  straighten 
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oul  the  hoist  diain.  The  requirement  for  task  lighting  was  particularly  important  in 
matching  the  missile  canister  guides  to  the  missile  rack  rails.  Lighting  was  also  generally 
in  detaching,  attaching,  and  inspecting  the  missile  canister  cables. 


2.  Common  task  difficulties:  The  task  of  aligning  the  "LT  shaped  guides  on  the 
bottom  of  the  missile  canister  with  the  supporting  rail  of  the  missile  rack  required  the 
most  lighting,  appeared  to  be  the  most  difficult,  causing  delays  and  requiring  repeated 
attempts  for  successful  completion.  Alignment  of  the  three  canister  guides  with  the 
support  rail  required  that  crewmembers  repeatedly  lift  and  realign  the  147  lb  canister  to 
close  tolerances.  This  concern  with  lighting  requirements  and  fine  alignment  tasks 
should  have  implications  for  the  NLOS  Full  Scale  Development  system  where  one  man 
working  on  top  of  the  vehicle  will  align  233  lb.  missiles. 


3.  Observations; 

a.  Qipping  a  Red  Filter  flashlight  onto  web  gear  while  performing  tasks 
produced  a  swinging/flashing  red  beacon  like  a  train  crossing  signal  to  the  Night  Vision 
equipped  camera.  May  need  to  emphasize,  or  introduce  as  SOP,  that  crewmen  use  their 
flashlights  in  the  “push  button  for  on*  mode  whenever  possible,  rather  tlian  leaving  the 
light  on  during  task  performance. 

b.  Alignment  of  the  missile  canister  guides  with  the  supporting  missile  rack  rail 
was  an  awkward  task  requiring  the  aewman  to  use  one  hand  to  aim  the  flashlight  while 
lifting  and  aligning  the  missile  canister  with  his  free  arm  and  shoulder.  What  are  the 
implications  for  a  233  lb  missile  one  man  on  vehicle  alignment  task. 

c  The  standard  flashlight  with  Red  Filter  made  critical  map  information 
(artillery  impact  areas)  di^layed  in  red  disappear.  TSM  NLOS  says  that  printed  red 
map  information  should  appear  under  the  r^  filter  condition,  but  that  the  red 
information  on  the  map  u^  in  the  evaluation  might  have  been  a  local  over-stamp  which 
would  disappear. 

d.  With  realistic  hands-on  training  crewmen  were  able  to  perform  NLOS  reload 
tasks  such  as  brush  guard  renKwal  simply  by  feel  with  no  illumination  at  all.  This  finding 
reinforces  the  value  of  equipment  bay  night  operations  training. 

e.  The  NLOS  lOE  systems  Gunner’s  Console  produced  a  great  deal  of 
illuminatioa  Gunner’s  Console  illumination  created  a  glare  on  the  windshield  that 
prevented  the  gunner  from  seeing  a  person  standing  as  close  as  the  front  bumper  of  the 
vehicle. 


f.  Night  reload  appears  to  have  been  less  dangerous  due  to  the  technique  where 
the  ground  man  handing  up  the  hoist  would  hold  his  flashlight  to  illuminate  the  on- 
vehide  man’s  two  handed  task  of  putting  the  hoist  dolly  on  the  boom  rail.  Training 
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developers  might  wish  to  do  motor  pool  night  operations  evaluations  with  the  FSD 
system  to  identify  such  detailed  aspects  of  task  assignment  that  can  promote  task 
performance  safety. 

g.  New  reload  system  tasks  might  require  soldiers  to  communicate  clearly  to 
coordinate  fine  crane  movement  adjustments.  This  could  be  a  concern  with  one  man  on 
the  ground  operating  the  crane  handle  and  the  other  on  the  vehicle  guiding  the  front  of 
the  233  lb.  canister  into  position  at  the  front  of  the  support  frame. 

h.  A  cluster  map  light  may  be  right  for  the  crew  cab  and  reload  tasks  because 
all  the  tasks  are  within  arms  reach.  The  crewmembers  stated  that  the  CML  lights  will 
not  replace  the  standard  issue  flashlight  because  some  tasks  require  longer  light  focus 
and  greater  illumination. 

i.  Can  we  identify  a  fix  for  the  existing  equipment  (standard  issue  flashlight) 
rather  than  acquiring  high  cost,  limited  utility  low  signature  cluster  map  light  equipment? 
Is  there  a  way  to  attach  an  opaque  low  signature  diffusion  filter  on  the  standard  issue 
flashlight  to  illuminate  tasks  wifiiin  hands  reach  while  eliminating  the  long  focus  and  hot 
spot  created  by  transparent  filters  and  the  flashlight  mirror  focus  distance. 


6.  The  PCX}  for  this  action  is  Mr.  Bill  Sanders,  AV978-4491/5297. 


WILLIAM  SANDERS 
Research  Psychologist 
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ATTACHMEOT  #1:  EATTLE  DRHI5  FOR  NON-LTNE-OF-SICSHT  FIBEK  OPTICS  GUIDED 
mSSIIZ  SYSTEM  (NLDS  FOG-M)  VERSION  7  (18  MAY  1989) :  SINGLE  MISSILE  RELOAD. 


Aaaiafnt  Qunaiir 

1.  CoBBUidt  'Prapar*  for  Hlttlla  1.  Acknowlsdgaa . 

Raload. * 


*2.  Dlaaounts  vahlela  and  Inatalls 
vahlcla  chocks  at  roar  whaal  on  tha 
•unnar's  slda. 


a2.  Raaovas  SINCCAR's 
hand  sat,  dtsBounts 
vahlela,  and  Inatalls 
vahlela  chocks  at  rear 
wheal  on  tha  assistant 
gunner* a  aide. 


3.  Reaovas  umbilical  brush  guard. 


3.  Assists. 


4.  Reaovas  umbilical,  fiber  optic  4.  Mounts  vehicle, 

connectors  and  quick  disconnect  pin.  unseats  missile  canister. 


5.  Retrieves  HHE  and  hands  to  Asst.  5.  Assembles  MHE. 

Gunner. 


6.  Removes  antenna. 


6.  Dismounts  vehicle. 


7.  Remove  emptr  can  by  hand.  If  can  7.  Assists. 

Is  full,  remove  using  KHK  and  sling. 

8.  Attaches  missile  sling  around 
missile  can,  C.G. 

9.  Align  can  with  launch  rail.  9.  Raises  missile  can 

level  with  launcher. 


10.  Pushes  missile  Into  launcher. 


10.  Adjusts  tension  on 
missile  sling. 


11.  Removes  sllng^ seats  missile  can. 

12.  Installs  umbilical  cable,  fiber  12.  Swings  MHE  to  front 

optic  connector,  and  quick  disconnect  pin.  of  launcher. 


14.  Stows  MHE. 


17.  Installs  umbilical  brush  guard. 

*18.  Removes  vehicle  chocks  from 
the  gunner's  side. 


13.  Mounts  top  of 
vehicle. 

14.  Dismantles  MHE. 

15.  Dismounts  vehicle. 

16.  Installs  antenna. 

17.  Assists  with  brush 
guard. 

*18.  Removes  vehicle 
chocks  from  the  assistant 
gunner’s  side. 

19.  Installs  SIMCOAR’s 
hand  set. 


20.  Announces  'Missile  Reload  Complete.* 

EHD  or  BATTLE  DRILL  NO.  B 
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ATTACHMENT  #2:  UGHTING  INTERVffiW  FORM  QUESTIONS  AND 
CREWMEMBER  RESPONSES 


1.  Did  implementation  of  any  Secure  Lighting  method  degrade  your  capability  to  do 
your  job? 

Answer:  "No  problems" 

2.  Did  you  e]q)erience  any  problems  with  ai^  aspect  of  the  man-machine  interface?  If 
so,  please  describe  the  situation. 

An^er:  "No  problems" 

3.  Were  you  able  to  distinguish  color  information  from  all  color  coded  sources  (using 
CML  lights)  lights,  indicators,  etc.? 

Answer:  "Yes" 

4.  Did  operators  e]q)erience  any  problems  due  to  not  being  able  to  see  or  read  any 
display,  warning  label,  etc.?  If  so,  describe. 

Answer:  "No  problems" 

5.  Was  there  ample  lighting  to  perform  operator  and  maintenance  tasks  in  all  work 
areas?  Describe  inadequacies. 

Answer:  "lighting  level  was  OK,  crews  could  do  PMCS" 

6.  Were  all  visual  alarms  (i.e.,  failure  lights,  easily  detected)?  If  not  describe  problems 
encountered. 

Answer:  No  visual  alarms  were  presented  to  crews.  Crews  stated  that  there  would  not 
be  a  problem  seeing  visual  alarms. 

7.  Was  the  meaning  of  indicator  lights/legends,  meter  values,  or  labels  understandable? 
If  not,  describe  any  difficulties. 

Answer:  "No  problems  anticipated" 

8.  Did  you  have  any  problems  march  ordering  or  emplacing  the  ^tem?  If  so,  please 
describe  the  problenL 

Answer:  March  Order  and  Emplacement  tasks  were  not  performed 

9.  Were  there  any  problems  e]q>erienced  with  energizing/de-energizing  or  initializing 
any  equipment  item?  If  so,  what  problems  were  there?  (Evaluation  was  limited  to 
power-up,  map  reading,  and  reload  tasks) 

Answer:  "No  problems" 

10.  Did  the  Ouster  Map  light  (CML)  or  other  light  sources  provide  adequate 
illumination  to  enable  performance  of  your  tasks? 

Answer:  "Yes,  the  Ch^  flashlights  and  Clip-On  lights,  and  the  Red  and  Blue-Green 
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filter  flashlights  all  provided  enough  light  to  perform  tasks” 

11.  Which  switch  position  (illumination  level)  did  you  find  most  adequate/effective  for 
various  tasks? 

Answers:  Gunner  #1  1/2  power.  Gunner  #2  1/2  power,  Gunner  #3  3/4  power.  Gunner 
#4  full  power) 

12.  What  was  the  most  suitable  distance  to  position  the  lights  to  acconqilish  particular 
viewing/reading  tasks? 

Answers:  Gunner  #1  14",  Gunner  #2  12",  Gunner  #3  6",  Gunner  #4  6" 

13.  Was  the  CML  or  other  light  sources  illumination  cone  too  small  or  too  large? 
Answer  Three  gunners  said  they  were  fine,  one  said  they  should  be  wider) 

14.  Where  did  you  mount  the  light  while  in  use?  (Be  specific)  Control  Panel  >  Oothing 
•  handles  -  cab  structure. 

Answer:  The  question  is  specifically  referring  to  the  CML  Clip-On  light  Gunners 
stated  that  CVda  should  be  located  on  the  radio  mount  below  dash  level,  with  one  for 
the  driver  and  one  for  the  gunner. 

15.  Where  did  you  mount/stow  the  light  when  not  being  used? 

-Was  the  light  cable  length  satisfactory? 

-Was  the  light  cable  cumbersome  or  tothersome? 

-Was  the  PWR  hookup  point  satisfactory? 

-Which  power  hookup  method  is  most  acceptable  (direct  wiring  or 
cormect  plug)? 

Answer:  This  question  was  not  ai^licable  because  the  CML  Qip-On  light  was  not 
stowed,  and  the  light  cable  hook-up  was  a  tenq>oraiy  arrangement  using  small  jumper 
cables  to  take  power  off  the  vehicle  batteries. 

16.  Was  the  clip  too  large  or  too  small? 

-Was  the  spring  tension  adequate/excessive? 

(Tension  should  be  greater,  it  would  ^de  off  in  off  road  use) 

-Was  utilization  capability/manipulation  satisfactory? 

Answer  Gurmers  said  the  dip  was  ^e  right  size,  but  t^t  spring  tension  would  have  to 
be  increased  and  a  rubber  lip  added  to  the  attadunent  dip  so  that  it  would  not  shake 
loose  during  vehide  movement  Utilization  was  satisfactory. 

17.  Was  the  light  effective  in  illuminating  control  panels  to  permit  adequate  viewing  of 
controls  and  displays  to  accomplish  all  required  operations? 

Answer  "No  problems” 

18.  Was  the  light  effective  for  reading/viewing  TMTs,  miqrs  and/or  other  material? 
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Answer  All  four  light  sources  were  considered  here.  The  Red  Filter  flashlight  made 
red  danger  zones  disappear  during  a  map  reading  task.  The  Blue-Green  Filter  made 
(blue)  water  markings  dis£q;>pear.  Both  CML  flashlight  and  Clip-On  light  provided 
accurate  color  representation  during  nuq>  reading. 

19.  Was  the  light  used  for  other  purposes?  If  so  describe. 

Answer:  light  was  not  used  for  other  purpt^es. 

20.  Did  the  light  cause  excessive  glare?  noticeable  -  disturbing? 

Answer:  No  glare  problems  with  the  CML  lights,  the  glare  from  the  Gunner’s  Console 
limited  forward  vision  out  the  windshield  so  that  the  gunner  couldn’t  see  a  man  standing 
at  the  front  bumper. 

21.  Did  the  light  iUumination/methods  employed  for  use  interfere  with  the  use  of  the 
night  vision  goggles? 

Answer:  This  was  not  evaluated. 

22.  How  does  use  of  the  illumination  provided  the  CML  compare  with  alternative 
illumination  methods? 

Answer:  CML  provides  softer,  less  bright,  with  good  color  rendition.  Gunners  said  they 
would  like  both  the  CML  flashlight  and  CML  Oip-On  light 

23.  Did  you  encoimter  ai^  other  difficulties  in  performing  tasks  using  the  light? 

Answer:  No  difficulties  were  observed. 

24.  Have  you  suggestions  on  how  the  light/iUumination  level  might  be  improved? 
Answer:  Gunners  agreed  that  the  mercuiy  switch  should  be  removed.  Gunners  may  not 
have  been  aware  that  the  mercuiy  switch  feature  could  be  turned  off. 

Added  questions: 

25.  Design  of  the  CML  flashlight 

Response:  Change  the  CML  flashlight  to  a  standard  flashlight  configuration  by  adding 
the  attachment  clip  and  turning  the  lens  head  90  degrees.  Take  out  the  mercury  switch. 

26.  Prefer  fixed  CML  versus  flashlight 

Response:  For  in  the  cab  and  reload  tasks  the  CML  Qip-On  and  flashlight  are  better 
because  of  their  lower  signature,  but  the  standard  flashlight  must  be  retained  for  other 
tasks. 

27.  Need  a  GI  flashlight  besides  the  CML  fixed  or  CML  flashlight 

Response:  Yes,  CML  is  good  for  tasks  at  hands-reach,  but  it  will  not  illuminate  more 
than  a  few  feet  Definitely  need  a  standard  flashlight  in  addition  to  the  CML. 
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